Addressing the Mapping Problem in Sonic Information Design through Embodied Image Schemata, Conceptual Metaphors, and Conceptual Blending
(2019)
author(s): Stephen Roddy
published in: Journal of Sonic Studies
This article explores the mapping problem in parameter mapping sonification: the problem of how to map data to sound in a way that conveys meaning to the listener. We contend that this problem can be addressed by considering the implied conceptual framing of data–to–sound mapping strategies with a particular focus on how such frameworks may be informed by embodied cognition research and theories of conceptual metaphor. To this end, we discuss two examples of data-driven musical pieces which are informed by models from embodied cognition, followed by a more detailed case study of a sonic information design mapping strategy for a large-scale Internet of Things (IoT) network.
Between Data and Breath: Machine Learning, Musical Embodiment and the Emergence of Voice
(last edited: 2026)
author(s): Jonathan Reus
This exposition is in progress and its share status is: visible to all.
From vocal deepfakes to artificial voice actors and pop star avatars, data-driven machine learning has intensified embodied, musical, and social complexities of voice. While disembodiment and decontextualisation of voice have been musical concerns since the invention of sound recording, AI voice synthesis accelerates these processes and adds new perceptual, cognitive, and social layers.
Many ontologies from voice studies imagine voice as resisting fixity, yet in today’s technological climate this resistance may be losing its ontological imperative. Voice is in transformation - possibly crisis - requiring both curiosity and care in paradoxical tension. These changes also unfold within a technological arms race for innovation, profit, and global AI supremacy. Artists are not only early adopters, but experimentalists and bards who participate in the narratives around AI and vocality.
This thesis evaluates the changing vocal condition through first-person artistic research with AI voice technologies, exploring their poetics and potentials in three artworks created between 2021–2025. In Search of Good Ancestors / Ahnen in Arbeit was a year-long generative radio broadcast exploring machine learning as a intergenerational vocal memory. iː ɡoʊ weɪ is a hybrid extended voice performance practice using real-time voice transfer to unravel vocal identity on stage. DadaSets investigates the invisibilized vocal labour of AI voice through collaborations with artists, new scoring systems, the absurdist dataset-making performance Bla Blavatar vs Jaap Blonk, and the invention of the voice synthesis instrument Tungnaá.
These works are analyzed through an interdisciplinary lens: experimental vocal traditions and the embodied musical-technological ethos of STEIM, alongside philosophies of voice, cognitive neuroscience, and material anthropology; while predictive coding theory frames compositional notions of uncanny, pathological and convivial technologisations of voice. Voice data emerges as paradoxical - both disembodied and relational, material and emergent, gift and commodity - functioning as the basis for musical animacy and collaboration within a rapidly changing socio-technical landscape.
This article explores the mapping problem in parameter mapping sonification: the problem of how to map data to sound in a way that conveys meaning to the listener. We contend that this problem can be addressed by considering the implied conceptual framing of data–to–sound mapping strategies with a particular focus on how such frameworks may be informed by embodied cognition research and theories of conceptual metaphor. To this end, we discuss two examples of data-driven musical pieces which are informed by models from embodied cognition, followed by a more detailed case study of a sonic information design mapping strategy for a large-scale Internet of Things (IoT) network.open exposition comments (0)