Research Catalogue

In the ongoing collaborative work Sensory Cartographies, Sissel Tonn and Jonathan Reus explore the relationships between consciousness, body and environment, as mediated by mechanisms and sense technologies.

Data-driven, realistic and identity-bearing AI voice technologies have been proliferating widely in recent years. Voice, a multiply embodied phenomenon situated within and across human bodies in space, is deeply disrupted by the disembodying tendencies of AI voice technologies and their processes of data creation and collection, resulting in the need for a re-evaluation of perceptual, cognitive and cultural factors. This paper addresses this need by synthesizing ideas from embodied cognition, voice studies, and material anthropology to analyze real-time, AI-mediated voice as a form of embodied cognition that is an enactive, intersubjective, extended, materially and socially distributed phenomenon. We examine AI-mediated voice through the case study of the live music performance iː ɡoʊ weɪ, which uses real-time AI voice transfer systems trained on carefully curated vocal datasets that represent diverse forms of vocal alterity relations. The live performance system integrates custom RAVE models within a SuperCollider environment, enabling dynamic real-time transformation of the performer's voice through a tactile control interface. This technical architecture facilitates immediate feedback loops between biological vocalization, AI processing, and auditory perception. The domain of audience perception becomes a key point of reflection, where the formation of new perceptions of voice and body is repeatedly challenged by the fluid voice-body gestalt of the performer, a process of donning vocal masks that modulate perceptual dissonances and resolutions. We further address the complex situation of voice AI through the concept of identity-bearing technologies, which leverages theories from voice studies, speech science, material anthropology and embodied cognition that address identity perception. Finally, we address the key ethical dilemmas of such systems, particularly as they relate to the problem of designing technologies that reproduce the illusion of a singular essential vocal identity. We respond to the problem of the representation of vocal bodies through a speculative framework of the vocal gift, analyzing how gift relationships are at play within the creation of the AI systems of iː ɡoʊ weɪ, and suggesting further directions for research into developing technological systems that honor gift relationships as a fundamental principle for ethical design of AI voice technologies.

From vocal deepfakes to artificial voice actors and pop star avatars, data-driven machine learning has intensified embodied, musical, and social complexities of voice. While disembodiment and decontextualisation of voice have been musical concerns since the invention of sound recording, AI voice synthesis accelerates these processes and adds new perceptual, cognitive, and social layers. Many ontologies from voice studies imagine voice as resisting fixity, yet in today’s technological climate this resistance may be losing its ontological imperative. Voice is in transformation - possibly crisis - requiring both curiosity and care in paradoxical tension. These changes also unfold within a technological arms race for innovation, profit, and global AI supremacy. Artists are not only early adopters, but experimentalists and bards who participate in the narratives around AI and vocality. This thesis evaluates the changing vocal condition through first-person artistic research with AI voice technologies, exploring their poetics and potentials in three artworks created between 2021–2025. In Search of Good Ancestors / Ahnen in Arbeit was a year-long generative radio broadcast exploring machine learning as a intergenerational vocal memory. iː ɡoʊ weɪ is a hybrid extended voice performance practice using real-time voice transfer to unravel vocal identity on stage. DadaSets investigates the invisibilized vocal labour of AI voice through collaborations with artists, new scoring systems, the absurdist dataset-making performance Bla Blavatar vs Jaap Blonk, and the invention of the voice synthesis instrument Tungnaá. These works are analyzed through an interdisciplinary lens: experimental vocal traditions and the embodied musical-technological ethos of STEIM, alongside philosophies of voice, cognitive neuroscience, and material anthropology; while predictive coding theory frames compositional notions of uncanny, pathological and convivial technologisations of voice. Voice data emerges as paradoxical - both disembodied and relational, material and emergent, gift and commodity - functioning as the basis for musical animacy and collaboration within a rapidly changing socio-technical landscape.

Discourse and materials pertaining to the experimental iteration with Jonathan Reus, during the Algorithms that Matter (Almat) artistic research project.