Motion, Music, Mediation: Bridging Tradition and Technology in Swedish Folk Dance-Music

Olof Misgeld

Introduction

This exposition presents the outcomes of a research project, where I, as part of a group of Swedish folk musicians and dancers, use optical motion capture (mocap) as a tool to explore our joint performance and to create new artistic output by combining traditional practice with contemporary media technologies. As a musician who plays for dancers and works in collaboration with the dancers I play for, I explore ways to mediate polska dance through the sonification and visualisation of data captured from the moving bodies. The exposition centres around live folk music and dance combined with sonic and visual displays of movements recorded from our performances in the mocap studio. I view such combinations as additional interfaces for exploring the tacit knowledge situated in the interactive polska music–dance practice and for the artistic purpose of extending the traditional art form into new forms of expression. 

The research addresses two key questions: how movement sonification and visualisation can illuminate the articulation of polska rhythms in dance and music, and how such mediations can create an augmented experience of the polska music–dance interplay in performance contexts beyond the participatory social dance. Approaching these questions, this exposition engages with the mediation of mocap-recorded dance as a novel interface for experiencing the rhythmic-metric framework in polska music and dance. The methodology involves conducting and documenting performances in the mocap studio, designing sonification and visual displays of movement data and evaluating such mediations through interactions within the polska music and dance practice.

The primary perspective of the research comes from my role as a dance musician, for which I am solely responsible for designing and conducting the research and interpreting the results. However, the dancers’ contributions involve their co-creating in performances, providing their recorded movement data, reflections, and interview responses at various stages of the research. Although music is the dominant perspective, the outcomes therefore shed light on a shared, interactive music and dance practice. The investigation focuses on the fundamental connection between sound and movement. Previous works (Misgeld and Holzapfel 2018; Misgeld, Holzapfel, and Ahlbäck 2019; Misgeld and others 2021) have used music and dance theoretical concepts by Ahlbäck (2003) and Blom (1981; 1993) to analyse such music and dance interactions. These analytical concepts are here utilised in the design of movement mediation and applied and tested in an artistic context.

Optical motion capture (mocap) is a technique that uses infrared cameras to record the positions of reflective markers placed on bodies and objects. In the entertainment industry, mocap is frequently used to project the performance of human actors onto virtual characters, thereby imbuing digital creations with the elusive authenticity of actual human performance. In contrast, this work builds on our ability to recognise human actions even when they are mediated through limited streams of movement data, such as single marker movements on a dancing body. A previous study (Misgeld, Holzapfel, and Ahlbäck 2019) examined how musicians can align their playing to point-light visualisations of selected dance movements displayed on a computer screen; I here apply these findings to visualise and sonify dance movements in an artistic context.

Sonification encompasses a broad range of practices for translating data into sound (Hermann and others 2011; Dubus and Bresin 2013), including utilising the human capacity to detect time-based patterns in acoustic signals. Sonification applications are typically influenced by the intent of conveying data relationships in forms that align with specific purposes. For instance, somatic sonification (Giomi 2020) represents interactive sonification practices that seek to enhance embodied awareness in physical activities, such as dance, sports, or rehabilitation. In artistic contexts, sonification can refer more broadly to generating sonic output from data for diverse artistic purposes (Polotti and Goina 2021). In this work, I use sonification to create transparent mappings from movement data to sound, facilitating the experience of movement and sound interactions in the context of folk music and dance. The tool SonifyFOLK (Misgeld, Lindetorp, and Holzapfel 2023), designed for accessible sonification of folk dance movement data, serves as the basis for all sonic outputs in this exposition.

By analogy with how dance-music denotes forms of music intended or performed for dancing, one could imagine the term music–dance for dancing strongly connected to particular forms of music performance, such as with the Swedish polska. This exposition revolves around the performance, Dancing Dots — The Exhibition (first performed 14 September 2022), which showcases a combination of live music and dancing, along with sounds and light displays generated from dance movement data captured through optical mocap. In the piece Dancing Dots, sonified polska dance is treated as dance-music — by listening, dancing, and playing to it in ways reflecting the traditional forms of the practice. With these performance interactions, the aim is to develop new ways of artistic expression in Swedish folk music and dance and, through the mediation of dance movements into digitally created sound worlds, offer new entrances for experiencing this art form. As such, this bridging across performance contexts involves hybrid forms of pre-recorded, mediated, and live performance, whereby the roles of interaction in the traditional setting are challenged, and the traditional, participatory polska dance is staged in an immersive performance space and extended visual/sonic landscape. Thus, the project’s overarching theme is to explore concepts for an open-ended development of artistic expression with traditional practice both as a departure point and a primary source of inspiration.

The exposition is organised into sections covering various aspects of the project. ‘The Video Abstract’ gives a non-verbal overview of the process and performative results. The ‘Polska’ section provides historical context and explains essential concepts related to the dance and music, such as svikt and asymmetric beat. ‘Towards a Mediation of Dance’ presents a background on transmitting and interpreting dance through sonified motion capture data. ‘Sonification — From Motion Capture to Dance-Music’ explains the process of collecting, selecting, and processing movement data, mapping data to audio, and interpreting the sonic results with examples of playing to dance sonifications. The ‘Visualisation’ section displays animations of the same data selection used in the sonification and discusses how these visualisations relate to the polska dance. The ‘Dancing Dots — The Exhibition’ section showcases various approaches towards combining live dancing and playing with sonified/visualised dance in the documentation of performances. Finally, the ‘Discussion’ section summarises and discusses the project’s results.

The participants in this work, including the author, the dancers Ami Dregelid and Andreas Berchtold, and the musician and sound/light artist Hadrian Prett, are established performers and pedagogues in the Swedish folk music and dance community and employed within central educational institutions in the field, such as the Department of Folk Music at the Royal College of Music in Stockholm (KMH), the Folk Dance Education at Uniarts in Stockholm, and the Eric Sahlström Institute in Tobo, Uppland, Sweden. The research was conducted as part of the doctoral project ‘Music-Dance Mediations — Performative Explorations into an Asymmetric Type of the Swedish Polska’ (Misgeld 2024) at KTH Royal Institute of Technology, Stockholm, Sweden.