Maxime Mantovani 2/5 : Meeting with Philippe Esling
As part of his artistic research residency, Maxime Mantovani is developing an organic control interface for a sound synthesis engine using artificial intelligence models. A proposal that came at the right time, says Philippe Esling, head of the research team working with the composer.
When Maxime's proposal first reached us," recalls researcher Philippe Esling, "it was perfectly in line with our scientific preoccupations at the time: to develop a meaningful control for artificial intelligence. Today, either the machine works by itself, almost autonomously (even if it only does what we ask it to do), or it generates a waveform that follows a source signal. An organological gesture associated with artificial intelligence is sorely lacking."
One of the main challenges is generating sound in real-time.
Philippe Esling is joined by doctoral student Antoine Caillon and post-doc Axel Chemla-Romeu-Santos in the team which relies on models they have already developed, notably the timbre transfer model. From a source sound signal, the machine generates a sound that follows the same profile (or at least some selected parameters of this profile), but in a different timbre (for example: at the input, a violin melody, at the output, a singing voice), following exactly the same profile, and even the expressive contours of the musical line, adapting them to the destination timbre. Then, instead of a sound signal, the machine will have to be able to interpret the information imparted to it by the musician, via the control interface developed by Maxime Mantovani and Emmanuel Fléty.
"Maxime's involvement in the team and his feedback on the different models he tests means that we can test these models and their sound qualities, better determine their usability limits, as well as their latency (the time needed for the machine to generate the new speech according to the input, the idea being to get as close as possible to instantaneousness), and therefore develop new ones, much faster than usual," explains Philippe Esling.
Maxime's contribution was unique and without his remarkable work it would have been impossible to obtain generation models of the musical quality we have today.
Photo 1 : Philippe Esling
Photo 2 : Maxime Mantovani dans les studios de l'ircam
