In the dense jungle of research projects taking place at IRCAM, the project Sébastien Gaxie is conducting with the Sound Analysis and Synthesis team is truly one-of-a-kind. While most residencies revolve around sound processing, creation and diffusion or how it interacts with other art forms, “Inside the Rainforest” bridges the gap between music and another science which, though far removed from the concerns of music, shares its focus on sound: the study of the Amazonian forest’s biodiversity.

Like many good research stories do, this story came to be by the purest form of serendipity. In 2021, Sébastien Gaxie and photographer Vincent Fournier embarked on one of those fanciful projects he is so fond of: an imaginary bestiary. In the way of Jules Verne, acting thus as both writer and musician, Gaxie imagines in Auctus Animalis a scientific expedition on its journey to discover the fauna and flora of an unknown island located in the middle of the Pacific Ocean. This work, narrated by comedian Denis Lavant, was awarded the Swiss Life Price.

La forêt amazonienne Guyannaise © DR

However, just like Verne before him, our dreamer wishes to make his make-believe story as scientifically accurate as possible. That is how he came to work with Jérôme Sueur, lecturer at the Muséum national d’Histoire naturelle in Paris, researcher in biodiversity and founder of a new scientific field called “ecoacoustics”, to create the cry of his imaginary chimeras. This encounter was significant, in that it opened up a wide range of possibilities for the composer: biophonic sounds, which qualify all sounds produced by animals, and geophonic sounds – referring to those produced by the elements. These two groups differ from anthropophonic sounds, which cover all man-made and mechanical sounds, such as… music. Jérôme Sueur’s research covers many topics, which include projects focusing on monitoring biodiversity by listening to and analyzing natural soundscapes, such as forests. One such projects is taking place in the Nouragues research station, located deep inside the Amazonian forest, in French Guiana. Sébastien Gaxie is immediately fascinated by this operation and its principle, which is diabolically simple but offers potentially endless possibilities. The ecoacousticians installed microphones deep in the forest, at a distance from the research station, where the local fauna wouldn’t be disturbed. One of the mics was placed at human height, close to the ground, while the other was put above the treetops, at a height of 40 meters. Both of these captors simultaneously record for one minute every fifteen minutes, all day long, every day, for more than five years; a priceless treasure of data for the ecoacousticians who intend to use it to study the behavior of animals, the functioning of the ecosystem and how it could potentially be disrupted over time. However, one challenge stands in the way of the path seemingly perfectly mapped out by the researchers: how to deal with this data? How can they harvest useful material for their research and use it to draw conclusions, both statistical and biological? That question is what gave Sébastien Gaxie the idea of a collaboration with IRCAM. The institute’s researchers, who specialize in sound capture and analysis and signal processing, are best placed to work on designing tools, with the assistance of machine-learning, to help the biologists in their quest to capture the secrets of this data jungle.

Station de recherche des Nouragues, site d'Inselberg © DR

At Gaxie’s request, Rémi Mignot from the Sound Analysis and Synthesis team has already developed a simple software program called Tinamou, a name he borrowed from the ground-dwelling bird from Central America, known for its melodious song. This software facilitates temporal navigation within the collection of sound samples and automatically generates for each a sonogram – that is to say a spectrogram that plots the sound’s frequency against time. To the initiated eye, this sonogram transforms into a full score that every specie plays a part in within its own frequency field, without ever overlapping that of another species, or at least not at the same time of the day. Some very simple patches that were until now mainly used for computer-assisted composition were repurposed to refine this raw data. Diemo Schwartz, from the Sound Music Movement Interaction team, repurposed the concatenative real-time sound synthesis system CataRT, developed 20 years ago at IRCAM, into a new tool able to respond to the needs of ecoacoustics. This new tool segments the whole dataset into short, 2-seconds long extracts, each represented by a scatter plot organized by audio descriptors. “It is like flying over the database”, illustrates Sébastien Gaxie. While still rudimentary, these two tools are already able to single out a bird song, a monkey scream, or any cry – which helps the researchers identify the sound and tag it, so that the machine can in turn learn it. These developments are what led the students of Nicolas Obin – a researcher at IRCAM and lecturer at Sorbonne University – to develop Ecosurf, an automated environment for data prospection and visualization, which builds on the AI model SurfPerch, trained for the classification of bird’s songs. “We feed Ecosuf the sound recording of an animal and the algorithm explores the database to identify all the sounds samples that feature similarities with the recording”, explains Gaxie. It is an efficient tool to draw valuable statistics from the database, which the researchers can in turn use to model the circadian and seasonal cycles of the sounds produced by the animals. A vital contribution to ecoacoustics!  

Ircam Eigen Mike © Ircam-Centre Pompidou, photo : Philippe Barbosa

This massive amount of data is also a treasure for Gaxie, who finds in it a fascinating sound material that is as ever-changing as it is breathtaking. To him, it seems like nature has composed true masterpieces of sound, that are as smooth and inventive as the best works of the greatest names in music. While Gaxie immediately sees the compositional potential of this project, this raider of the lost sound wants to go a step further. He decides to join an expedition to the research station to see with his own eyes and hear with his own ears the marvels that he has, up until now, only heard in recordings. He also uses this opportunity to make longer recordings, with an Ambisonics system featuring a 35-capsule Eigenmike. Over the span of ten days, he captures fifteen hours of recording. Once back in Paris, he plays some excerpts, using IRCAM’s Ambisonics dome, for Pierre Forget[1], ecologist at the Musée national d’Histoire naturelle. As someone who has been a regular visitor to the Nouragues station for 40 years and has just come back from a mission, Forget is amazed by the accuracy of the result, which he finds very similar to what can actually be heard in the forest. Rarely has an immersive sound diffusion experience been so powerful and efficient. Just by closing your eyes, you are thrown into the middle of the Amazonian forest; you shiver to the sound of the tropical rain, you try to swat a mosquito away, you delight at the harmonies of the Tinamou’s song or that of the tawny-bellied screech owl’s, you are captivated by the hypnotic power of the howler monkeys… Yet all these sounds combined, far from becoming a cacophony, create instead a stunning formal clarity. “This new sound space struck me as a kind of tabula rasa, it’s a true cosmogony”, concludes Sébastien Gaxie. Now, he dreams of using this material in his next work to create a dialogue between this wonderful biophony and the orchestra – a conversation between the tropical forest and art. To take up this compositional challenge, he will require yet again the help of the tools developed at IRCAM. For instance, patches for the spatialization software Panoramix enable users to navigate in the Ambisonics space, which allows them to single out the sound from an animal or, on the contrary, play with the fascinating polyphonies that only biodiversity can create.

Interview by Jérémie Szpirglas