Exposition

The Data-driven Voice-Body in Performance: AI Voices as Materials, Mediators, and Gifts (last edited: 2025)

JC Reus
Jonathan Reus
open exposition

About this exposition

Data-driven, realistic and identity-bearing AI voice technologies have been proliferating widely in recent years. Voice, a multiply embodied phenomenon situated within and across human bodies in space, is deeply disrupted by the disembodying tendencies of AI voice technologies and their processes of data creation and collection, resulting in the need for a re-evaluation of perceptual, cognitive and cultural factors. This paper addresses this need by synthesizing ideas from embodied cognition, voice studies, and material anthropology to analyze real-time, AI-mediated voice as a form of embodied cognition that is an enactive, intersubjective, extended, materially and socially distributed phenomenon. We examine AI-mediated voice through the case study of the live music performance iː ɡoʊ weɪ, which uses real-time AI voice transfer systems trained on carefully curated vocal datasets that represent diverse forms of vocal alterity relations. The live performance system integrates custom RAVE models within a SuperCollider environment, enabling dynamic real-time transformation of the performer's voice through a tactile control interface. This technical architecture facilitates immediate feedback loops between biological vocalization, AI processing, and auditory perception. The domain of audience perception becomes a key point of reflection, where the formation of new perceptions of voice and body is repeatedly challenged by the fluid voice-body gestalt of the performer, a process of donning vocal masks that modulate perceptual dissonances and resolutions. We further address the complex situation of voice AI through the concept of identity-bearing technologies, which leverages theories from voice studies, speech science, material anthropology and embodied cognition that address identity perception. Finally, we address the key ethical dilemmas of such systems, particularly as they relate to the problem of designing technologies that reproduce the illusion of a singular essential vocal identity. We respond to the problem of the representation of vocal bodies through a speculative framework of the vocal gift, analyzing how gift relationships are at play within the creation of the AI systems of iː ɡoʊ weɪ, and suggesting further directions for research into developing technological systems that honor gift relationships as a fundamental principle for ethical design of AI voice technologies.
typeresearch exposition
keywordsvoice, voice and live electronics, uncanny, AI, convivial technologies of voice, live extended voice performance, real-time voice transformation, embodied voice, sound poetry, extended voice, voice studies, performance studies, embodied interaction, music technology
date20/09/2024
last modified16/09/2025
statusin progress
share statuspublic
affiliationSussex Digital Humanities Lab / EMUTE Lab (University of Sussex), Intelligent Instruments Lab (University of Iceland)
copyright(C) Jonathan Reus
licenseCC BY
languageEnglish
urlhttps://www.researchcatalogue.net/view/3799783/3799784
external linkhttps://jonathanreus.com/portfolio/igouwei/


Copyrights

name250812-igouwei--250815-morph-export
copyrightJonathan Reus
licenseAll rights reserved
usagesvideo
nameAIMC-2023-igouwei-2otherspecies
copyrightJonathan Reus
licenseAll rights reserved
usagesvideo
nameigouwei-signal-flow-diagram.drawio
copyrightJonathan Reus
licenseAll rights reserved
usagespicture
nameigouwei-score-interface-2
copyrightJonathan Reus
licenseAll rights reserved
usagespicture
nameigouwei-interface-2
copyrightJonathan Reus
licenseAll rights reserved
usagespicture
namejvox95x1_04
copyrightJonathan Reus
licenseAll rights reserved
usagesaudio
nameblonk2_95x1_04
copyrightJonathan Reus
licenseAll rights reserved
usagesaudio
nameanimal2x2_11
copyrightJonathan Reus
licenseAll rights reserved
usagesaudio
namechorala2x2_16
copyrightJonathan Reus
licenseAll rights reserved
usagesaudio
nameacry03vae10x2_21
copyrightJonathan Reus
licenseAll rights reserved
usagesaudio

Comments are only available for registered users.