EarToy

Sound Avatars and Real-Time Musical Interactions

The EarToy project studies hearing/body/space interaction in immersive sound reproduction systems. The EarToy project uses 3D audio reproduction technology and motion capture to explore the possibilities of hearing/body/space interaction through different scenarios, applied primarily to the domain of musical playing. The goal is to promote interactive applications prioritizing the sensorial auditory modality and to develop new paradigms for interaction with sound. The creation of interactive environments that focus on the auditory modality raise interesting scientific issues in the field of spatial auditory cognition and offer the potential for the conception of innovative applications in the music world.

It has been suggested that work focus on the concept of a sound avatar, an auditory manifestation of the user in a virtual world. One of EarToy's objectives is to elaborate a catalogue of interaction modes with auditory settings that change according to the user's movements. This phase relies on the construction of publishing tools for interaction making it possible to describe, using symbols, the rules of association between a category of gestures and the associated procedures of transformation and spatialization of a sound corpus. The pertinence at a perceptive and cognitive level of this catalogue of interactions will be assessed through experiments.

The concepts and techniques developed will be assessed in the framework of two pilot applications: mobile mixing and interactive access to music.

Project Details

Participants