How do we let ourselves be carried along, or navigate, within the fabric of sound? What perceptual and cognitive processes govern listening and musical interaction? These are among the questions that Emmanuel Ponsot, Patrick Susini and Clément Canonne have been exploring together for the past three years, devising increasingly inventive experimental setups to test their hypotheses, both scientifically and musically. Their idea stemmed from the “cocktail party effect.” First described in 1953 by British cognitive scientist Colin Cherry, this effect refers to the ability of our auditory system, from the outer ear to the brain, to focus on a single voice within a noisy environment.

“Unlike in the visual domain, where focusing attention on a particular object usually allows us to isolate it easily, in hearing everything overlaps,” explains Emmanuel Ponsot. “And yet our auditory system manages to decode what we are listening to in a rather remarkable way. Listening to polyphonic music presents a fairly similar situation, but it had never really been examined from this perspective.”

“This raises a number of questions,” continues Patrick Susini. “What do we actually listen to within a sound scene? Do we perceive it as a whole, or attend to its details, and why? How do we navigate within a sound scene, and how do we shift from one mode of listening to another? How does an event within that scene, for instance a screech in an urban environment, capture our attention?”

By adapting and rethinking methods from psychoacoustics, the three researchers have devised listening situations in which they can control different parameters and make attentional strategies measurable. At the same time, they aim to better understand the specifically musical, or creative, dimension of listening itself, and how it brings out salient features of the musical discourse.

Benjamin Matuszewski et Clément Canonne lors du concert interactif, Espace de projection, Ircam, 2026 - Photo © Quentin Chevrier

In 2023, they devised a two-part concert-laboratory. The first part focuses on improvised music. Two clarinetists take part in an improvisation that is not entirely free, but follows an interactional framework defined in advance. The hierarchical relationship between them evolves over time: each musician knows that at certain moments they must take the lead, step back, or engage on equal terms. The same applies to timbre, which they are asked either to blend or to contrast.

During roughly twenty minutes of improvisation, all combinations of these parameters are explored. Two solo passages, one for each musician, serve both as moments of musical breathing space and as control conditions for measurement. The audience, meanwhile, connects via smartphone to a web application developed by Benjamin Matuszewski from the Sound Music Movement Interaction team. The app takes the form of a simple slider. Throughout the performance, listeners move it to the left, center or right depending on where their attention is focused.

The second part involves composer Sébastien Roux, who created a series of 32 miniatures using algorithmic processes. Certain rhythmic patterns are generated with a Python library, while a Max program controls the evolution of the musical material. Each miniature is a three-voice electroacoustic piece, spatialized over three loudspeakers. Each one functions as an auditory tracking task. Listeners are asked to follow a specific voice and, at the end of each excerpt, to indicate whether they detected a brief glissando or vibrato affecting that voice.

The constraints of composition allow the researchers to manipulate several variables. First, spatial segregation, whether the sound sources are co-located or distributed in space, and temporal segregation, whether the texture is homorhythmic or, on the contrary, non-overlapping. The hypothesis is that greater separation in space or time makes it easier to follow a single voice.

Another line of inquiry concerns salience. When listening to polyphonic textures, one may alternate between two modes. A “local” mode, focusing on a single stream and possibly switching between streams, and a more “global” mode, attentive to the overall result of fusion or separation. These modes do not exclude one another and may alternate continuously.

In this context, salience refers to the capacity of a stream to attract attention, even when it is not the focus. The researchers distinguish between event-based salience, for example a louder note or an acoustic deviation, and structural salience, which relates to the internal organization of a stream, whether it is highly active or, on the contrary, more continuous and smoother.

Concert interactif, avec l’Ensemble ]h[iatus sur des pièces composées par Sébastien Roux et Clément Canonne, Espace de projection, Ircam, 2026 - Photo © Quentin Chevrier

At the end of this concert-laboratory, the results came in. In the first experiment, findings were mixed regarding timbral fusion and contrast, but much clearer concerning the hierarchical relationships between the improvisers, which had a direct impact on attention. In the second experiment, spatial or temporal separation led to a success rate about 20% higher than in conditions without separation, a significant effect. For salience, detection reached around 80% for event-based salience and nearly 100% for structural salience. A highly active or overly “discursive” stream can interfere with selective listening. The researchers’ hypotheses were therefore confirmed.

“It should be acknowledged that composers and performers did not wait for us to make use of these phenomena,” notes Clément Canonne. “In Bach’s counterpoint, for example, one already finds forms of temporal separation. And polychoral practices, such as those at St Mark’s Basilica in Venice, clearly explore spatial separation.”

At the same time, no experimental protocol is without limitations. “Interacting with a web application while listening is not a normal concert situation,” Canonne adds. “It changes the experience, as does being observed or adopting a reflective stance. Our experimental conditions themselves influence attention. Balancing artistic freedom with scientific rigor is not straightforward. The repetition inherent in experimental design can even produce a certain fatigue, which runs counter to the spontaneity of live performance, despite the high quality of the music.”

Concert interactif, avec l’Ensemble ]h[iatus sur des pièces composées par Sébastien Roux et Clément Canonne, Espace de projection, Ircam, 2026 - Photo © Quentin Chevrier

Like any experiment, this one suggests directions for future work. This led to an interactive concert on March 18, 2026, as part of the Ateliers du Forum, performed by the ensemble ]h[iatus. “This is no longer a concert-laboratory, but an aesthetic experience, playful and distinctive,” says Canonne. “We may still use some of the data, but the primary goal is musical performance and enjoyment. The slider from the first experiment inspired us to create situations in which the listening experience itself becomes part of the piece. With Sébastien Roux, we imagined feedback loops between what the audience perceives and what the performers do.”

This is not a democratic system based on voting. The audience does influence the music, but implicitly. Their perception of the performance shapes the behaviour of the musicians in real time. Several aspects of perception come into play, including interaction, similarity and difference, thresholds, and duration.

For example, in one piece, the ensemble ]h[iatus improvises on short motifs by Sébastien Roux. After a first sequence, the audience is asked, via a web application developed by the ISMM team, to press a button for as long as they felt the improvisation lasted. The average perceived duration becomes the length of the next improvisation. If it is shorter, the tempo accelerates; if longer, it slows down. “We are playing with the relationship between perceived duration and experienced speed.” In another piece, two field recordings by Sébastien Roux, a highway and a schoolyard, are diffused in the space. Listeners press a button for as long as they perceive the recording. The resulting perception rate is transmitted in real time to the ensemble, which must keep it around 50% by adjusting its dynamics. The audience’s perception regulates the performance, which aims to maintain a threshold state of listening. “Here, the musical discourse is genuinely steered in real time by the audience.”

This situation, combining human and non-human sound sources, raises further questions. What is the effect of such heterogeneity on attention? This is the focus of a fourth experiment, planned for June 2026 during the ManiFeste festival, based on a commission from Raphaèle Biston for the augmented accordion duo Xamp and live electronics. To what extent does the electronic source compete with the instrumental one? How does its salience reshape the distribution of attention? Does it exert a kind of “magnetic pull”, continuing to attract attention even after it disappears? And does spatialization really have the power often attributed to it?

“These are fascinating questions,” says Clément Canonne. “This kind of experiment will teach us a great deal about how we perceive mixed music. Even without direct industrial applications, it may influence musical creation by shifting composers’ concerns.” As Patrick Susini notes, the musical expertise of listeners may also play a role, a question he plans to explore further. “We observe a great deal of variability in listening behaviour,” adds Emmanuel Ponsot, “and this freedom of attention is what gives polyphonic music its richness.”. Susini also points to possible medical applications: “A better understanding of attentional processes, and of the different forms of auditory salience, could help refine the settings of hearing aids in complex sound environments such as concerts.”

By Jérémie Szpirglas