Deep generative audio models and AI in MaxMSP and Ableton Live
As part of its research on deep generative models, the work of the ACIDS group in the Musical Representations team at IRCAM is expressed in the design of several cutting-edge AI tools for musical and creative synthesis. The goal is to provide novel tools to model musical creativity and extend sonic possibilities with machine learning approaches. In this context, the team experiment with deep AI models applied to creative materials, aiming to develop artificial creative intelligence. Over the past years, the ACIDS group developed several objects aiming to embed these researches directly as real-time objects usable in MaxMSP and Ableton Live. The team has produced many prototypes of innovative instruments and lightweight embedded deep audio models. The researchers now notably provide the RAVE VST, the nn~ library and the FlowSynth system.
Objectives
Upon completion of this course participants will be able to use the existing various generative AI tools environments, and notably to go further than the existing possibilities by training their own models. The goal is also to provide the basic notions allowing to include existing open-source deep models to their creative workflows.
Public
Composers, musicians, teachers, performers, researchers.
Prerequisites
- Participants must be comfortable with computer music environments (MaxMSP, Ableton Live) and experience in composing with these tools.
- A beginner's level knowledge of Max software is strongly recommended. If you are learning on your own, we suggest that you do at least the first 10 tutorials
- A beginner’s level knowledge of Python is strongly recommended.
- Knowledge of signal processing basics is a plus
- Experience in mathematics and computer programming is a plus
Training Period and Organization
30 hours of training. November 4-8, 2024. 10am-1pm/2:30pm-5:30pm
10 students maximum
In the event that IRCAM cannot physically welcome students for a program that requires face-to-face instruction, the program will be canceled.
Program
After a brief introduction on the concepts of artificial intelligence and machine learning, participants will dive successively into the mechanisms and the concrete use of nn~ and RAVE, both for using the models, but also being able to train their own models. All tools will be addressed both conceptually (mornings) and through practice (afternoon), in pairs and collectively. The last day of the training will be devoted to a full experimentation day.
Monday
| Morning | General introduction: - Core concepts of artificial intelligence and machine learning for music - Understand the basics of latent models and the notion of representation learning Existing models: Deep neural audio synthesis, introduction to real-time timbre transfer, RAVE: From Python to VST |
| Afternoon (RAVE VST) | Application and hands-on Use cases |
Tuesday
| Morning | Existing models (part 2): The nn~ library, developing your own creative workflow, how to train new deep models |
| Afternoon (nn~) | nn~: hands-on session in pairs nn~: training your own models |
Wednesday
| Morning | Embedding deep models in Max4Live - Using open-source deep models in Python - Setting up an OSC server between Max and Python - Basics of Max4Live devices |
| Afternoon (Deep M4L) | Environment and concepts Getting to grips with communication |
Thursday
| Morning | Extending the existing libraries - Discussion on core concepts of externals - Embedding new models inside the externals |
| Afternoon | Extension hands-on session: sharing, debriefing |
Friday
| Morning | Global tools, hands-on session |
| Afternoon | Hands-on session and general debriefing |
Educational Resources and Techniques
- Training room equipped with iMac computers with software, headphones, MIDI keyboards, and microphones. The hands-on sessions will take place in an IRCAM studio
- Technical requirements: each participant is asked to bring their own database of sounds (or several if need be) to work with RAVE and nn~. This will be used to feed the training mechanism in order to produce a new model. All data are good to use, regardless of the repertoire (personal music, completely exogenous material), but a set of sounds that are relatively homogeneous in terms of timbre and type of instruments will make it easier to train. The overall length for the combined set of all sounds is recommended to be over 2 hours of audio.
- For the hands-on improvisation sessions, we will form pairs. You will also be asked to use your own input source, which can be any instrument, synthesizer or acoustic source.
- Training format: alternating between theoretical explanations and hands-on exercises
- Teaching aids: video-projected presentations, course material with bibliographic references, software documentation
The software and plug-ins available in the Premium membership are not included in the training price and are not supplied at the end of the training.
Supervision
- Welcome the first day of training beginning at 9:45am
- Attendance controlled; signature of an attendance sheet required every morning and afternoon
- A certificate at the end of the program is given to each participant with the results of the exam
Assessment
- Assessment of knowledge in the form of an open-ended creative project.