Innovative combinations of digital and acoustic sounds are not uncommon in contemporary folk, traditional, and world music, and extensions and adaptions of instruments have contributed to the evolution of the genres in modern times. This work has explored the mediation of motion-captured movement to unveil performance practice and explore new ways for artistic expression in Swedish folk music and dance. An artistic goal has been to bridge across performance contexts by creating an immersive sonic and visual landscape out of a traditional music–dance performance. Sonification of dance becomes a new articulation of the music and dance practice with the potential for developing knowledge and understanding. This way, several results can be identified that are open for further exploration:
New artistic expressions
Sonification of movement opens a previously unexplored interface between dance and music in Swedish folk music, offering new entry points and dimensions for experiencing dance–music interaction and allowing new forms of interactions than in the traditional setup, such as performing to sounds derived from dancing. Movements manifested as sound gain substantiality in the audible domain and become an additional sonic layer to the dance-music. The digital sound worlds created by movement sonification open ways to experience folk music beyond traditional representations with acoustic instruments, tunes, and dances. The sound design in this project aims to maintain essential aspects of rhythm, metre, and tonality in order to facilitate real-time interactions between digital sounds and traditional performance. Furthermore, it exemplifies digitally created dance music with human, flexible, non-quantised timing. With this said, this work leaves areas open for further explorations of fields of tension between electro-acoustic and folk music aesthetics.
Unveiling practice
Sonifications of movement augment patterns in the dance as an audible complement to the metric/rhythmic fabric of the music performance. Such unveiling of previously unheard parts of the practice gives access to new layers of understanding of the music–dance relation, with the potential to inform the practice that it is based on and provide a basis for developing artistic expression.
From my experience as a musician in the present project, the sound augmentation of movements revealed details and relations that may not otherwise have been perceived. Examples are the rhythmic cycles of specific dance movements and the relations between the asymmetric metre in the music and the dance. Previous research on polska music has shown that variations of the polska metre, i.e. with a more or less asymmetric beat, are usually consistent with rhythmic gestures. That is to say that the degree of asymmetry in polska tunes changes with the melody. With the sonified dance, different rhythmic layers become present simultaneously through sonifying different movements. For example, in turning sections, sonification show more pronounced beat asymmetry. This is not to suggest that dancers do not continuously follow the beat asymmetry of the melody, but that the relation between asymmetric beat variations in music and dance is not simply one-to-one, as if the dancers at all instances would change movements strictly according to the beat variations implied by the melody. Rather these sonifications suggest that the dance and music, at some level, function as repositories for one another, allowing multiple simultaneous rhythmic/metric interpretations within the shared framework of the polska. Such an approach to movement sonification offers musicians additional ways to explore the variability of polska metre with attention to a multiplicity of rhythms in the dance movements. The project’s design has been influenced by theoretical concepts in polska music and dance, which have played a crucial role in the aesthetic choices made during the sonification and visualisation processes. For instance, the sonification of position and velocity data explores ‘svikt’ as an expression of ‘asymmetric’ polska metre. This exploration found that these and other concepts are valuable tools within this artistic context.
A tool for the arts
Movement data can be seen as a repository with potential for various interactions. In our case, dance movements are captured as data streams, and after processes of calculation, sonification and staging, they are re-mapped onto the bodies of the performers, as they are interpreting the sounds by performing to them. Through the open-ended character of these processes, sonified movement becomes a resource for interactions and interpretations through different practices. Karreman (2017) exemplified how recording technology has influenced performance practice, with actors becoming performance capture artists with the use of mocap in film and gaming. The sonifications in this work are rendered from recordings where the performers were unaware that the data would later be used for generating sound and, therefore, did not intend to adjust movements for that purpose. Recent research on historical performance has studied archival recordings by contemporary players’ recordings with early recording technology (Stanović and Billiet 2021). In related ways, sonification of dance movements could be applied to archival recordings for investigations into historical dance practice.
Challenge the performance context/the interaction
In Dancing Dots, the pre-recorded dance sonification imposes a temporal grid for the live performance. An exciting result of this is that although the dance sonification sounds were described by the dancers as highly relatable and recognisable, they felt utterly insensible towards their actions — as if coming from a ‘ghost’. Upon reviewing our performances, we noticed that our timing sometimes drifted in relation to the sonification and the moving lights. Still, the interaction between the live playing and dancing remained interlocked through these parts. In interviews, Ami Dregelid contrasts the discomfort of being unsure of her timing with the sonification to feeling how naturally things fall into place when dancing to live music. The interviews with dancers revealed how they simultaneously understood the sonification through recognising their own movements and interacting with sounds interpreted as dance-music and how they, in particular, favoured continuous sound models.
The multidimensionality of these experiences suggests further explorations of the affordances of dance sonification interactions in artistic and pedagogical settings. Although a successful mediation of the embodied experience of dancing was achieved to some degree, the dimension of the live interaction through shared timing resisted bridging the performance context. One route towards overcoming this would be to approach a real-time sonification with a sufficiently low degree of latency. Such an approach would further affect the roles in the interaction by, in effect, making the dancers instrumentalists. Other possible extensions of this work include capturing and sonifying other aspects of movement, such as the distribution of weight, to reach further into the embodied interactions of movements and music. To summarise, the mediation of motion-capture data has offered new perspectives on movement patterns in Swedish polska through abstractions, reductions, and spatial conversions, made accessible in sonic and visual renderings and interpretable through embodied, performative interactions.
