Maxime Mantovani 1/5 : Associating the organological gesture with AI
Maxime Mantovani's compositional approach is closely linked to new technologies. Very early on, he created his own computer tools and developed his own electro-acoustic lutheries designed to control sound in a personal and artistic way.
“It seems important to me to make electronic music with an instrument that allows me to create and perform it," he says. I want the body to be involved in the generation of sound. This comes from a certain culture of electroacoustics, from a desire to manipulate sound as closely as possible and to listen to it for itself, for what it evokes...".
A graduate of the CNSMD in Lyon, and a graduate of François Roux's class, in 2019 he joined the Cursus at Ircam, which enabled him to discover other approaches to composition, and in particular certain computer-aided) composition tools. In fact, for his Cursus graduation piece, Existentia, he uses a fragment of code in Python, "Audioguide", the result of a previous artistic research residency with Benjamin Hackbarth, in collaboration with Philippe Esling.
"Audioguide" is a program for sound analysis and re-synthesis by concatenation, a good example of what can be done in morphological transfer - that is, automatically generating a waveform according to a source, reproducing its profile but changing its timbre (for example, a violin as input, a flute as output).
It was also an opportunity for the composer to meet Philippe Esling and his team, giving rise to a common desire to advance the artificial intelligence tool: the objective of this residency is indeed to design an interface, hardware and software, specifically designed for the real-time control of artificial intelligence models, while ensuring reproducibility and strong expressiveness.
This implies the development of a real-time audio synthesis AI, which "learns" how to produce sound by feeding itself with a quantity of recorded waveforms.
But not only that. Since Maxime Mantovani's idea is to link this sound synthesis tool with the sound control interfaces he has been developing for a long time, in order, for example, to be able to truly interpret and improvise himself by controlling the sound synthesis in real time in an almost organic way - like any musical instrument, in fact.
So we need to improve and refine the control interface on the one hand, and the synthesis engine on the other.
Maxime Mantovani at ircam studios
"Concerning the control interface, I worked a lot with Emmanuel Fléty, who is the prototype engineer at Ircam: we started from the tool that I had developed alone so far , and we added screens to visualise certain parameters (such as the waveforms of the different sources), which means that we don't need a computer screen. We also modified the internal cards to increase the power, as well as the connectivity, to gain modularity.
In terms of artificial intelligence, our work focused on several areas. The first being data: all these sounds, organised by family and divided cut up, form what is known as a dataset, which the machine must be fed with so that after digestion by deep learning, it can synthesise some in turn. The sounds must obviously be of high quality, but also varied - which Maxime Mantovani achieves by inviting musicians to play: starting with Henri-Charles Caget, a former percussion teacher at the CNSMD in Lyon, who records a large number of Western and non-European percussion instruments for him, or the tuba player Victor Auffray on the flugabone.
"Neural networks designed by Antoine Caillon learn how these sounds are constructed by analysing the waveforms, creating what is called a pre-trained model. Then, using one of these pre-trained models, the machine can either imitate it by morphological transfer or generate sound automatically. This system is unique in the world, because it runs in real time, under Max!
Existentia by Maxime Mantovani (recorded as part of the Cursus concert) Centre Pompidou, 2020
In the next few episodes, we will meet the members of the research team to discover the challenges of developing artificial intelligence for this new system and we will plunge into one of Maxime Mantovani's improvised music sessions with musicians on "acoustic" instruments.
