Methodology
In order to develop an ideal method to gather information about musicians’ movements on stage, two main aspects should be taken into account: characteristics and setting of these movements and desirable nature of the resulting data in light of possible future applications.
Regarding the first aspect, one should be aware that musicians’ movements whilst playing are not free, but deeply tied to the characteristics of the instrument being played. Jensenius’ (2007) distinction between sound-producing, ancillary and sound-accompanying movement is here very relevant, as the expressive character should be looked for especially amongst the last two categories.
With organ playing being my main personal interest, and no previous literature which has been produced on this particular instrument, it seems logical to refer to the findings related to expressive movements in piano playing, which is the most similar playing condition. In this context, the most interesting data on expressive body movements have been found in the body parts that are not directly involved in the sound production, such as the upper body with neck and shoulder (Camurri et al. 2003), the head (Dahl & Friberg, 2007), the elbows and the hands when not playing (Thompson & Luck, 2012). Moreover, the musical characteristics of the piece that is being played also have an influence on the amount of freedom that is left to the performer in their movements, which is higher in the technically less-challenging sections, moments of stillness and musical breaks or rubato sections (Davidson, 2007). Overall, it should be considered that musicians’ expressive movements whilst playing might be quite small and detailed, which is why a high precision is desirable when detecting and analysing them.
Regarding the output of the analysis, it should be appropriate for both scientific and artistic applications, in order for this knowledge to be available to a broader community. Whereas statistical analysis often proves to be useful to determine the nature and size of relationships between phenomena, uniquely numerical results might therefore be less accessible and applicable in most artistic fields and, vice-versa, solely abstract or metaphorical interpretations of the data are arguably not suitable for applications in a scientific/psychological context. The aim of this study is therefore to find a method which results are solid and simple enough to be translated in the language that is most appropriate to the field of interest.
Besides music, movement analysis is fundamental to many other fields, such as sports, dance, theatre and linguistics. In these contexts, extensive research has been conducted, using various methods and instruments with a subsequent multitude of analysis and results that allow one to range from the most objective to the freer, metaphorical interpretations. In addition to the already mentioned literature focusing on music-making, the broad knowledge that has been produced in other fields represents an excellent source of information and inspiration. For this reason, two strategies of data collection and analysis on movement are going to be presented in their applications to various phenomena, in order to examine their characteristics and evaluate their possible application for our purpose.
Motion capture technology
The motion capture technology is an increasingly common way to store and investigate music-related movements, especially in psychological/cognitive studies. This system is based on a series of (infrared) video-cameras and strobe lights, capturing markers that are attached to the moving subjects. The coordinates of these markers can be the digitalized and used for analysis. More innovative systems are also being developed for markerless motion capture, producing three-dimensional representations of subjects by using dynamic algorithms to group pixels of the same kind.
Having acquired time-series data representing the position of the markers, it is possible to derive various kinematic features of the performed movements, such as velocity, acceleration and jerk, defined as the rate of change of acceleration.
This kind of approach has been used in several experiments. In sports, it is often used, amongst others, to analyse gait (Rucco et al., 2020), transition play (Eom & Schutz, 1992) and improve performance (Hughes & Franks, 2004); it is also commonly used in dance, to develop training systems (Chan et al., 2010), studying the interpersonal dynamics of learning (Hegarini, & Syakur, 2016) as well as emotion recognition in dance movements (Burger et al., 2013). In the field of music, and particularly focusing on the influence of musicians’ movements on emotional communication, Castellano et al. (2008) have applied automated video analysis to the videos of a pianist playing an excerpt from Beethoven’s Sonata No. 4, Op. 102/1, with the heads and silhouette of the pianist being tracked. Results showed that the quantity of motion was not significantly influenced by the changes in the emotional conditions, except for the sad one. However, encouraging results were found in the relationship between the temporal peak of motion cues, with their attack and release, and the emotional expression in the performance. Velocity of head movements was also found to change significantly according to the different expressive conditions. The authors of this article have pointed out the smaller variations in movements due to the playing constraints, as well as the need to consider the relationship between movements and the music’s structural features.
In order to allow for a deeper analysis of such movement, algorithms and computational models have been developed, that can extract not only low-level motion features (position, amount of detected motion), but also movements’ expressive features (fluidity, heaviness) and the overall “tension” of the performance (Camurri et al., 2003). This represents the ideal link between the precision of computational, statistical data derived from motion capture, and the use of movement analysis in an artistic field, thanks to the broader, expressive-oriented nature of the analysis.
The EyesWeb experimental platform
The EyseWeb platform was initially created with the goal of modelling various composition and performance environments in terms of emotional agents (Camurri et al., 2020), with a particular interest in applying these agents to the communication through dance, visual media and music. Accordingly to this goal, it allows for the combined use of different technologies for the input of data, such as video cameras and body sensors, with a wide range of possible outputs that can therefore be applied to various fields. Since the beginning stages of the design of this platform, a particular focus has been dedicated to the history of movement analysis and choreography, with R. Laban’s theory of effort (1963) being one of the main sources of inspiration. This ideology can be still traced back in the constructing elements of the software and its “Gesture Processing library” (Camurri et al., 2003), for which reason I have decided to highlight and explain the possible links between the EyesWeb modules of analysis and the Laban expressive parameters that were described earlier in this text.
The Motion Analysis Library allows for the extraction of the following expressive parameters:
- Quantity of motion (QoM): defined as the total amount of detected movement. It is computed as the normalised area of a Silhouette Motion Image (SMI), which is obtained at frame t by adding together the silhouettes in the previous n frames and subtracting the one at frame t, so that current position is not considered (Castellano et al., 2008). As a function of t, the association to the corresponding Laban parameter of Time is obvious, but also that with Weight and Flow, which both depend on time-based measures such as velocity, acceleration and frequency.
- Silhouette shape/orientation of body parts: consisting in the ellipse which main axes represent the orientation of the body. This is a measure of the body’s relationship with its surroundings and can therefore be considered to contribute to the Laban parameter of Space.
- Contraction index (CI): calculated as the eccentricity of the above-mentioned ellipse or as the comparison between the area of the minimum rectangle surrounding the subjects’ body and the area of the silhouette. It is therefore a measure from 0 to 1 of how the body uses the space surrounding it, and whether the limbs are expanded or contracted compared to the centre. This last point is also part of Laban’s definition of Space, to which this measure can therefore be associated.
The Space Analysis Library is based on the division of into active cells forming a grid, in which a moving point is tracked. It allows to identify:
- Meaningful regions of space: regions in which movement is more “meaningful” or intense, also related to other features such as QoM. The association with the Space parameter from Laban’s theory is here very clear.
- Relationships with objects: defined as virtual, semantic or expressive spaces. This can contribute to the definition of Laban’s Weight, which has often been metaphorically described in terms of picking up or carrying heavy or light objects.
Trajectories and their features can be identified in 2D, resulting into the Trajectory Analysis Library:
- Length of trajectory: both in spatial and temporal terms, with an association to Laban’s Space and Time parameters.
- Direction: describes how the subject moves on the stage, an important determinant on the LMA’s Space parameter. It also indicates movements’ evolution in space according to the basis axes, which Laban component of Shape focuses on.
- Directedness index: how much a trajectory is direct or flexible. Ratio between the length of the straight line connecting the first and last point of a given trajectory and the sum of the lengths of each segment constituting the given trajectory. This clarifies whether a subject moves directly towards a goal or is freer in space, which is a central part of Laban’s definition of Space; in other terms, it also distinguishes between the movement being bound (to a goal or object) or free (Flow).
- Velocity (time). Integral to Laban’s definition of Time and Weight; also contributing to the Flow as a measure of movement’s tension.
- Acceleration (trajectory/space). Integral to Laban’s definition of Time and Weight; also contributing to the Flow as a measure of movement’s tension.
- Curvature. Describes movement’s relationship with Space as well as its Shape.
[back]
FIGURE 1, 2: Castellano et al. (2008) - automated tracking of the head and silhouette of the pianist
Laban Technique
Man’s fascination in the relationship between music and meaning can be dated back to the end of the 19th century, with personalities such as C. Darwin in the field of natural history and F. Delsarte for oratory. However, the field of movement analysis finally established itself around the 60’s, when people started to understand how body movements could also be very important carriers of truth and identity, somehow even more than verbal messages, and therefore felt the need to formalize their study. Developed by Rudolf Laban (1879-1958) and his pupils, the Laban technique for Movement Analysis (LMA) was soon applied in multiple fields such as dance choreography and criticism, performance scholarship, sports and dance therapy, as well as in corporate management (Lamb & Watson, 1979) and clinical psychology (I. Bartenieff, 1980; Martha Davis, 1984).
According to Laban (Laban & Ullman, 1960), the dynamic, expressive qualities of movement can be described through the components of effort and shape.
The concept of effort can be observed in the tension, release and phrasing of body movement (Maletic, 1987) and is subsequently divided in four motion factors:
- Weight: ranges from strong to light and can also be described in terms of velocity and acceleration as indications of pressure in the movement.
- Time: defined as how long it takes to perform an action, which can be sudden or sustained. It is therefore related to the speed, tempo changes and the flow of movements.
- Space: distinguishes whether the trajectory between two points is direct (focused on the goal) or indirect (interacting with other elements of space). It also recognizes a contraction or expansion of the limbs compared to the centre of the body.
- Flow: highlights the amount of tension in movement, that can be bound or free. It looks at the frequency of motion, its fluency and freedom.
These factors can be combined with each other, giving rise to more complex effort drives that are comparable to different kinds to expressive movements (Bartenieff, 1980).
In addition to the movement factors, representing the intention that is behind the movement, the component of shape describes their evolution in space according to the basis axes:
- Vertical: rising or sinking
- Horizontal: widening or narrowing
- Sagittal: advancing or retreating
Since the very beginning, the scientific field has often been reluctant in recognizing this system’s reliability in an empirical context, with their scepticism regarding mainly the possibility to replicate the descriptions of movement qualities and dynamics as well as the perceptual power of observers. This critique still rises nowadays, but it has not stopped the growth in the use of the Laban Movement Analysis in fields such as factory labour, robotics and therapy. A more recent study by Bernadet et al. (2019) has confirmed the reliability of LMA to be impacted by the differences in the selected observers/raters as well as in the type of movement that is being analysed. This might become even more evident in the study of musicians’ movements on stage, which relies on very detailed observations of likely small movements. However, the broad, flexible character of this technique is certainly the key of its success in many fields, and is highly desirable in light of the interdisciplinary goals of the current study.
FIGURE 3: Camurri, Mazzarino & Volpe (2004) - sub-regions of the body silhouette and body center of gravity
[back]