Astérismes 4/4 : The System’s Compositional Constraints
During her artistic research residency on a a distributed sound diffusion device "Astérisms" with researcher Jean-Philippe Lambert, composer Aki Ito has produced a few pieces that are used both for the duo's experiments and for the elaboration of a new musical language that is intrinsically linked to this lutherie-in-the-making. As is often the case in the field of artistic research, artistic and scientific developments go hand in hand, feeding each other with their related constraints, sources of creativity.
Latency, mother of all vices
Among the technical constraints in the Astérismes device, there is one that the duo suspected the existence of well before launching into their actual research: latency. Latency is the delay between the command to broadcast a sound, given to a terminal by the musician/composer, and its actual broadcast by the terminal in question. Knowing latency is essential, especially in the context of musical writing: how can we coordinate a speech if we don't know the reaction time of each voice?
Latency is a sum of several fragments of time, some of which are more easily controlled than others. The simplest comparison is that of an orchestra: in an orchestra, all the musicians can see the conductor directly and immediately. The conductor coordinates events through his baton. Now imagine that each musician could only see the conductor via a screen, and that the delay in the diffusion of the image was different for each one: this would lead to cacophony.
In the case of a set of cell phones, the role of the conductor is to synchronize the clocks of all the cell phones in play. Because each terminal has its own clock, but it is not always perfectly on time: some go too fast, others too slowly. Fortunately, Jean-Philippe Lambert had already tackled this problem in a previous project at IRCAM (CoSiMa) and had developed several strategies to solve it inspired by the way our terminals already synchronize themselves, on a daily basis, with an online atomic clock. Phew.
The Asterism project team in Ircam's studios
On the other hand, there is another delay that is much more difficult to control, which could be compared to the one linked to the attack modes specific to each musical instrument: the sound of a violin, for example, is emitted more quickly than that of a clarinet... Musicians obviously know, almost instinctively, these delays and take them into account when they want to play together. In this case, it is the delay between the moment when the order is received by a telephone to play a sound [1], and the moment when the browser of this telephone actually plays it.
And this delay is a priori different for almost every smartphone model. While web regulators have recently asked browser manufacturers and developers to specify it (for the benefit of web programmers, to be precise), for the moment only a tiny minority comply. And when, like the composer Aki Ito, one wants to add instrumentalists to the distributed sound diffusion of Astérismes, the stakes are even higher!
It is crucial to find a way to estimate this latency. A first solution using gesture capture was considered for a while. "One of our ideas was based on an application that the ISMM team has developed, called "CoMo-Vox", says Jean-Philippe Lambert. "CoMo-Vox is a web application that teaches you how to beat time, and is very useful for kindergarten and primary school teachers who are required by recent ministerial reforms to lead choral singing activities with their pupils... In our case, at the beginning of the performance, when the spectators connect, we wanted to ask them to beat time to a given rhythm: by comparing the data from the sensors when the sound is emitted by their phone, we can deduce the latency, and therefore readjust the sound diffusion orders."
"We could even imagine reusing the same gesture later on," adds Aki. "But we have to admit that, for the moment, it's not convincing," deplores Jean-Philippe. "When several people beat the rhythm together, each with their own phone, they end up having to beat the rhythm with several simultaneous measurements (since each phone has its own latency), and the task becomes impossible."
The Asterism project team in Ircam's studios
For the moment, no satisfactory solution has been found to measure latency. Just as for an orchestra without a conductor, there is a great risk of falling into cacophony. Not knowing the latency forces the composer to proscribe any rhythmic writing that is too fine or chiseled. "It becomes a part of randomness that I accept so that the music can exist," she says. "But I can use it to my advantage. When I compose rhythmic patterns, I take into account (or even anticipate) possible echo effects. When the audience is well spread out in the space, it gives the feeling of a form of reverberation. For certain rhythmic patterns, I can also reduce the number of phones involved, and I can obviously compose music that floats a little."
To stream or not to stream
Another limitation of the system is that it is not possible for the time being to stream the sounds to all the terminals in the audience which would undoubtedly pose other problems of untimely disconnections or slow downloading. The current solution is to send the sounds in packets (limited to five to ten minutes in size, again by software constraints) to be triggered at the appropriate time.
"This represents a huge mental burden for the musician," says Jean-Philippe, "but we are currently working on a new system, with Soundworks, the underlying system developed by the ISMM team on which we rely, that would allow sound to be downloaded continuously. But I'm sure that will lead to other compositional constraints!"
Other important constraints: timbre and volume. The bandwidth of telephones (in terms of sound, that is) is quite limited. The bass, in particular, does not work. "You have to find solutions to make it audible anyway," says Aki, "you can play with an accumulation, or with the integration of small additional loudspeakers to the device which may limit the space."
An advantage or a disadvantage, as you like. It depends on the musical direction chosen!
[1] Remember that "Asterisms" uses for its deployment the web tools included in the Internet browsers of smartphones.
