The achievement of virtual embodiment, i.e., how gestural expression animates the embodied avatars interacting in a virtual environment (Maes et al. 2024), brought me to a deep reflection on gestural adaption (Calleja 2007) and sense of presence in the new immersive dimension of an XR performance space.
Musicians are used to adapting their gestures in different performance spaces and real situations where they enact (Varela et al. 1991) and interact (Noe 2005; Leman 2016; Sutil 2015). To give a practical example, in the case of a concert, musicians have to formulate their playing of an instrument given the intended sound they wish to reproduce in a specific room acoustics, which is the physical, objective performance space. Moreover, while playing, they have to interpret a score or improvise by shaping their lived emotions and feelings in relation to the subjective performance space they are interacting with. We should take into account that the performer's subjective approach is biased by prevailing cultural conventions, which are also related to the empathic responses of the audience in a performance space.
This scheme summarises how a social frame determines the identity of a performance space, the nature and purpose of spectators' behaviours, and how they should be interpreted. The space where a performance is executed implies different modalities of perception both by the performers, who are the 'actors', and by the spectators, who are not passive recipients (Goffman 1974; Bourriaud 1998. From these perspectives, the performance space entails three different dimensions:
- an objective dimension as a physical space in which a performance takes place (e.g., a practice room, a concert hall, a theatre, etc.)
- an objective dimension as a physical space in which a subjective dimension unfolds, (i.e. the physical condition and lived experience of a performer within the physical space)
- an objective dimension, as an interactive space that entails the relationship between performers and audience
The way a performer lives in and interacts with a performance space is thus characterised by these conventions and cultural practices. Performers are engaged in conditions related not only to the physical space but also to the spatiality of sensations and situations (Merleau-Ponty 1945) that cause different sensorimotor predictive actions and gestural approaches. It is clear that traditional performance spaces and the interaction with the audience create different subjective and emotional circumstances, and different situational components, in response to circumstantial live performative contingencies (Doğantan-Dack 2012; Nijs 2017).
When the performance space is new, and the social frame is unfamiliar, the question is how a new performance space-frame will emerge, and what it implies in terms of involvement from the part of the performers and the audience. A performance in XR can provoke difficulties in keeping an empathic connection with a real audience because both performers and spectators lose the feeling of being part of a group in a common real performance space. In the expanded dimension of a virtual and immersive space, sensory information and perceptions are remodulated using a different proprioception (Adamovich et al. 2009; Vladimirovich et al. 2021)
While playing in the virtual environment, I experienced the adaptation of my proprioception in relation to an unconventional performance scene and a phygital condition. The connection with the physical world was always very present due to my contact with the real piano keyboard and the awareness of an audience in presence, but the general sensation was to play 'like in the water', immersed in the new flow of the virtual horizon. Looking at my embodied avatar after my motion recording, I noticed, for example, the necessity to remodulate some movements of my body to obtain a more explicit choreography. To facilitate the real-time interaction with my avatar in performance, I decided to emphasise the movements of my head nod, with respect to my solo practice in a real condition (Filippelli 2025: 298-299). However, the expedient of 'augmented' expressions is often applied in real chamber music performances to find a better connection (Bishop & Goebl 2017).
This means that in my XR performance, I searched for a real interaction with my avatar by adapting my gestures to this new condition of a virtual space, where my body interacts with a digital entity in a live performance. The contrapuntal dialogue with my avatar was expressed through the corporeal movements ensured by the reproduction of the whole body that allowed a realistic simulation. The corporeal interaction gave me a sense of engagement in the performance, as it was with a real performer, albeit without a facial expression.
The dichotomy between the absence/presence of bodies performing in a virtual environment is a recent territory of creative experimentation (Di Bernardi 2018), documentation (Bleeker 2017) and knowledge archiving and transmission. Although over the past thirty years, many scientific and artistic experimentations on choreutic and music practice have been drawn to study multimodality in digital performance (Fernandes 2016), its ontology has still to be defined. Nowadays, virtual stages— like broadcasts from television studios or hybrid XR networked places—are places where live performances can be attended, other than the traditional performance spaces, such as theatres and concert halls. These places are equipped with cutting-edge and specialised devices that can provide the reproducibility of a live performance. In this perspective, the concept of 'live event' also includes mass reproduction, in the language of media, and indicates forms of entertainment that can take place in real-time at the same time in different performance spaces, real and virtual. Qualifications such as live broadcasting and live recording are assigned in contexts that report different ways of communicative authenticity compared to the 'retransmission' of data. An example, in the musical field, is the live recording of a disc or a CD during a concert in theatres or concert halls, in front of a real audience. However, the production of audio-video recordings is often reworked in studios before being released. Therefore, they are pre-processed, fixed and retransmitted upon request. In contrast, television broadcasts and today's online streaming are also referred to as 'live' when sounds or images are captured and retransmitted without any editing, regardless of the manipulations made in the production studio or postproduction phase. Although the philosopher Walter Benjamin in his essay The Work of Art in the Age of Mechanical Reproduction from 1936 (2008) criticised the idea of art reproducibility as a risk to preserve its 'authenticity' (the artistic 'aura'), he also reflected on how it holds possibility for the (re)construction and the foundation of a social artistic frame and tradition. In the digital era, technology allows reproducibility and, seen in the form of art archiving, transmission and dissemination, this stimulates, at a social level, a reflexive awareness of the expressive and aesthetic qualities of the arts that, otherwise, would not exist (Heinich 1983).
This is also the case of an XR music performance, where gesture can be tracked and creatively transformed and reproduced in visuals, rather than being archived as pure motion data, useful for different scientific studies and purposes.
During my experimentation, both my motion recordings and visualisations served various purposes:
- to create and improve my performance
- to explore and study the gestural interaction in extended conditions
- to discover new modalities of motion collection, archiving, transformation and dissemination, also in 'live' reproduction
All these data sets are useful inputs for future applications and a witness to our contemporary 'techno-aesthetics' (Montani 2021). Playing in an XR performance, therefore, required adaptations to a new space-frame, with the awareness that this implies the discovery of new dimensions for human corporeal expressions and creative technological interactions.
Concerning the visual rendering, I chose how to built the visual space by following my tastes. In accordance with LWT3 team, my aesthetics were more oriented to be linked to Reich's music and avoid gaming representations. Fresh and warm colours of a minimal ambience were chosen and, for the same reason, the animation of the virtual pianists was reduced to a minimum resembling semi-abstract sculptures rather than cartoon figures. In the virtual stage, a micro-constellation of two particles systems were generated by the audio signal on the basis of the dynamics: one by the recorded audio track of the first virtual performer and one by the real-time audio track of me playing as a second performer.