Communicative movements: those that are used explicitly for communication between performers (or listeners).
Sound-accompanying movements: those that mimic or follow musical structure.
Ancillary movements: movements that are not directly involved in the sound production but are important for musical expression.
State of the Art
The types of movements that a performer makes on stage have been precisely classified according to their function, and are generally referred to as musical gestures, meaning “a movement or change in state that becomes marked as significant by an agent” (Gritten & King, 2006).
According to Jensenius (2007), these can be classified in:
In a further study, Thompson (2012) explains that these movement categories can be traced back to two bigger functions: music-producing movements, generally corresponding with the first ones in Jensenius’ categorization, and music-perceiving movements, often coinciding with the sound-accompanying ones and all music-induced movements. By clicking on the link for each type of movement, it is possible to access an overview of literature that has already been produced on the links between these, expressive communication between performer and audience, and musical properties.
Problematization
While the experience of a listener who plays music from a CD or from the Radio can be argued to be almost totally based on the musical discourse, and therefore described by the already-existing, ample literature on music perception and emotional communication (Sloboda & Juslin, 2001), the paradigm for audience experience in contexts where auditory information is not the only one being perceived might need further study and refinement. The evidence of people’s ability to infer information on the structural and emotional features of the music by watching music-producing and music-derived movement means that not only what they hear, but also what they see happening on stage has an influence on their overall experience of the performance. As such, one’s experience of the latter should be understood as a combination between the auditory and visual stimuli that the performance consists of.
Further studies could shed light on how performers’ movement features mediate emotional communication the audience, since the already-existing literature did not go further than comparing experiencing conditions with or without video, or static to exaggerated movements. This undoubtedly contributes to positively answer the question of whether visual stimuli influence music perception, but it does not explain how this happens, which kind of movements convey which kind of information and how the perception of the latter interacts with that of the music itself.
Moreover, it is interesting to reflect on the performer’s own movements, which certainly have the function of producing the sound, but might also be a result of the musician’s subjective experience of the piece they’re playing, therefore making the distinction between music-producing and music-perceiving movements not as clear. Furthermore, these movements have been shown to have the additional function of supporting the performer’s own affective experience (Bishop & Goebl, 2018; Dahl et al., 2010).), allowing them to reach a more solid interpretation by means of analysing, consolidating and more deeply connecting to the vitality of the piece. The notion of vitality contours is here fundamental, referring to the way in which an act is performed and the feeling that directs it (Stern, 1999). These vitality contours are not linked to the content (emotions, thoughts or actions), but to evolving patterns that are often described with dynamic kinetic terms, such as surging, fading-away, fleeting, explosive, tentative, effortful, accelerating, decelerating, climaxing, bursting, and drawn out (Stern, 1999, p. 68). They are at the basis of the appreciation of many arts, amongst which music, which also reveal multiple vitalities rather than a clear “content”; they are symbolically represented by the style of the player’s performing gestures, which in turn facilitate their expression and understanding both for the performer himself and the audience (Vanmaele, 2017).
The aim of the current study is to join the notions on music’s vitality contours and music-producing/music-deriving movement to further understand the dynamics of emotional consolidation and communication in the context of a music performance, investigating the following research question:
How do players’ movements relate to the inner vitality of the music?
In order to do so, however, it is necessary to find a method to appropriately categorize and describe such movements, collecting data that can be used in any subsequent analysis and/or artistic application. An intermediate research question is therefore:
How can musician’s movements whilst playing be captured, analysed and described, collecting appropriate data for a diverse array of applications?
Relevance
Investigating the correlation between musicians’ movements whilst playing and the characteristics of the piece that is being played can be a fundamental contribution to the already ample knowledge on music cognition and emotional communication during a music performance. Extensive studies have been conducted, deeply analysing the characteristics of gestures that are performed during speech (McNeil, 2005) as well as free dance (Van Dyck et al, 2014) or music-induced movements (Burger et al, 2012). All these studies have managed to confirm quite strong relationships between the characters of movements and those of the performer as well the audience’s emotional experience in the same moment, as well as interesting dynamics of support or contradiction between the two means of communication (Goldin-Meadow, 2003). However, whereas musicians’ movements have been proven to strongly influence people’s perception of the whole performance, the currently available literature doesn’t go in as much depth into describing such movements and explaining how they influence performer and audience experience in the practice. It would also be interesting to further explore this knowledge from the first-person perspective of a performer, especially focusing on ancillary- and sound-accompanying movements and how they contribute to the making of the performance before reaching the audience. Furthermore, the question of a “chronological” relationship in performers’ movements still remains open: can a distinction be made between music-producing and music-perceiving amongst the movements of the performer alone?
This leads to another point to which this study might contribute, that is musicians’ own awareness of their movements whilst playing and possibility to grow in it leading to better performances. Having exposed some music students to the Body Mapping (BMG) technique, designed to teach musicians skills in self-evaluation and change for performing with sensory-motor integrity, Buchanan and Hays (2014) showed how a deeper awareness of their body gave the students a higher ability to be poised and responsive, higher confidence in communicating musical ideas and improved self-awareness skills in making sensory connections with cognitive information, therefore producing more efficient movements. Kinaesthetic and simulated gesture of performing a piece have also seemed to inform musicians informalizing their thoughts (Minafra, 2021), with a deeper movement insight automatically generating deeper musical awareness and confidence in playing. Besides one’s own body awareness, it is also important to consider how the different elements of a performance, amongst which one’s visual appearance and body movements, contribute to making a perceptual environment that determines the audience’s experience.
Developing a solid method for capturing and analysing one’s movements whilst playing can provide helpful information for all these purposes, as well as remaining open for other kinds of applications. The development of music mediation technologies that use data from body movements to navigate and retrieve from music information libraries has long been one of the main goals in the field of embodied music cognition ((Godøy & Jensenius, 2009). Other than that, however, an understanding of these mechanisms and their categorization as a vocabulary of musical expressions opens the doors to a whole new set of possibilities of translation and transposition to other artistic contexts, fuelling new, interdisciplinary dialogues and creations.
Sound-producing actions: movements that produce the sound.
Studies have been conducted, using quantitative data and/or qualitative analysis of music-listeners’ movements to predict their subjective experience (Amelynck et al., 2012; Irrgang & Egermann, 2016) as well as the musical characteristics of the piece they were listening to (Irrgang et al., 2020). Amelynck et al. (2012), having collected accelerometer data of participants’ arm movements whilst listening to music together with the emotional ratings by the same participants as well as those of observers who were not listening to the music, managed to derive predictive models for valence and arousal. Although the results for valence were not as significant as those for arousal, this study confirmed the existence of close links between how one feels and how one moves while listening to music, as well as showing people’s ability to recognize the emotional character of music-derived movement.
Irrgang and Egermann (2016) expanded this approach to free body movements of participants who moved to a song whilst holding a mobile device, linked to their own ratings of the recognized emotions following the GEMS-9 scale, a model that was specifically designed in order to capture musically evoked emotions (Zentner et al., 2008) as a powerful alternative to multi-purpose scales that are based on non-musical areas of emotion research. Movement features such as size, rate, regularity and smoothness were found to be significant predictors for both arousal and valence. In accordance with the results of previous studies (Juslin & Laukka, 2013; Jordano et al., 2014), medium sound levels, high tempi and small timing variation predicted joy or happiness, while sadness was associated with irregular and slow movements. However, the difficulty of capturing the actual position in space through accelerometer data made it harder to capture some gestures and therefore to predict emotions such as nostalgia or transcendence that are less related to rhythm and more to gestalt features.
Not only music-induced or expressed emotion can be accurately predicted by movement, but also musical properties, as shown in a study by Irrgang et al. (2020). Motion features extracted from smartphone movement sensors were used to predict rhythmical properties of music; on the other end, pitch level and range were more difficult to predict, probably due to the more subjective answers that they evoke in terms of movement. Moderated rhythmical and structural complexity of the musical stimulus triggered the greatest amount of movement, while a high complexity generally made participants stop moving. In general, these results confirm that the dimension of arousal, more strongly related to rhythmicity, can be more easily predicted than valence, as previously reported (Amelynck et al., 2012; Camurri et al., 2004; Irrgang & Egermann, 2016).
The principal aim of these studies was to use the knowledge of the links between movement and music to develop systems for motion-based music retrieval, which overcome the potential barrier of language. However, this evidence might also have implications in the field of audience perception, especially in the context of live or video-recorded performances, as well as in the creation of multimodal artistic performances.
Numerous studies have shown that visual information plays a fundamental role in the way an audience experiences a musical performance (Coutinho & Scherer, 2017; Mitchell & MacDonald, 2016; Tsay, 2014; Vuoskoski et al., 2014). Audience evaluation appears to be shaped by what people see happening on stage, which can convey additional information on structural and physical qualities of the piece (Morrison et al., 2014; Schutz & Lipscomb, 2007), as well as its emotional character. The latter is, however, more problematic, with the two modalities not always conveying consistent emotional information (Ekman & Friesen, 1974; Chapados & Levitin, 2008; Vines et al., 2013).
Amongst all types of visual stimuli, performers’ body movements seem to be conveying the most information, as supported by Embodied Music Cognition’s awareness of the body as a fundamental mediator between music and mind (e.g. Bangert and Altenmüller 2003; Chen et al. 2008; see also Godøy 2011; Lahav et al. 2007; Maes et al. 2014). Already in 1872, in The Expression of Emotion in Man and Animals, Darwin described distinctive body postures and movements in response to emotion eliciting situations, showing the power of this body-mind connection both in humans and animals. Current research on the strong links between the execution of an action and its perception, which appear to be represented in the brain by the activation of similar functional regions, clearly explains how watching a certain movement can lead to the prediction of mental states such as feelings and emotions that modulate the perception of musical expressivity (Sedlmeier et al., 2011; Maes and Leman, 2013). Encouraging results have been found regarding the association of music and movement, which seem to share a unique dynamic structure when communicating emotion (Sievers et al., 2013). This movement-music coupling was shown to be cross-culturally valid and can be seen in a series of human behaviours. These are, amongst others, the correspondence between the deceleration of a runner coming to a stop and a musical ‘rallentando’ (Frieberg & Sundberg, 1999), as well as the similarity between common musical tempi and some biological rhythms such as human heartbeat (Moelents & Van Noorden, 1999). Possibly relying on the mirror neuron system of brain regions that activate both when perceiving and performing actions (Molnar-Szakacs & Overy, 2006), the deep structural links between music and movement might refer to the social and evolutionary function of music as a way of strengthening social bonds between humans (Phillips-Silver & Keller, 2012).