Designing From Listening: Embodied Experience and Sonic Interactions

An understanding of the richness of people’s sonic experience can lead to the creation of novel methods for informing design practices. One of the challenges in Sonic Interaction Design (SID) is to deal with the complexity of the “sonic”: its phenomenon, the interactions it creates, its social and cultural contexts. To tackle this challenge, this thesis investigates how we can draw upon people’s everyday sonic experience, particularly listening and remembering sound, to design interactions using body movement, digital sound processing and embodied technologies. Firstly, the research analyses how sound has been studied in its phenomenological, cultural and social aspects in fields such as Sound Studies and Embodied Sound Cognition. Secondly, it involves users in the process of designing sonic interactions, with a user study about gestural-sound relationships during active control of digital sound, and a series of participatory design workshops which draws upon people’s sonic experience for imagining interactions with sound. The thesis provides four main contributions. The first is Retro-Active Listening, a concept which draws attention to sounds heard in the past by remembering listening to them. The second is the Sonic Incident, a technique for SID workshops, which allows designers to explore participants’ past experiences of listening. The third is the Gestural Sound Toolkit, which enables designers to rapidly prototype interactive sound mappings based on human movement. The final contribution is three models for designing embodied sonic interactions. These comprise (1) Substitution, in which users’ movements substitute the cause of the sound, (2) Conduction, where users’ movements have a semantic relationship with the sound, and (3) Manipulation, in which users’ movements manipulate the sound. These contributions help to build a framework for design that addresses lesser-explored matters in SID, such as embodiment and contextual aspects of sound, which are potentially relevant for users.