Documenting our working sessions with video and high-quality audio capture is a powerful strategy because it allows us to quickly review and comment on our musical experiments. We have the option of re-listening several times, focusing on the ‘good bits’, and inventing new terms to describe these musical ideas. While new terms are always arbitrary at first, with repeated use they begin to offer a concrete, reliable way to refer to elusive musical ideas. We see these newly coined terms as empty containers that can be filled with meaning as we collaborate over time.

For example, the term ‘manifold’ came from a recent stimulated recall session. The term was inspired by the layered nature of the music we were experimenting with, but it eventually came to refer to a much more complex possibility space that we explored over the course of many video recordings. On the following page, we show additional examples of the terminology we created for manifold.

 

Certain choreographers in the field of dance have been using similar terminology-based methods for decades:

We began the aubiome project with the simple motivation to investigate various computer-saxophone ‘links’, with a focus on performance-related considerations rather than compositional ones. What kind of music could be produced if the electronics system is designed as part of the instrument, by the performer, rather than as part of a specific piece? We noticed a qualitative shift when we reconceptualized the system’s role as a platform for exploration and experimentation, rather than as a means by which a specific piece is realized.

Open-ended, exploratory working processes can reveal a wide range of potential outcomes, but not all of them are necessarily musically interesting. The musical explorer must be able to discriminate between the desirable and undesirable, making judgements about which path to take along the way. The experimentation process involves testing, judging and adapting, a method discussed by Joel Ryan:

Ryan describes how a composer works with technology to produce new music, but we have found that these issues are equally applicable to ‘hybrid instrument’ scenarios, in which a traditional instrument, such as the saxophone, comes into contact with real-time computer processing. The ‘good bits’ are often hidden among many less interesting options. In order to uncover them, we must adopt an exploratory mindset and have access to an appropriately-designed system. We wanted to be able to move quickly through the experimental cycle described above when designing the system for aubiome, and in particular, I wanted to be able to work hands-on with the saxophone for multiple cycles without interruption. We consider this capability to be an important aspect of our ‘performer-centric’ approach, because it prioritizes a workflow that leverages my expertise as a performer and puts me in a position to generate and explore my own musical material. In other words, this approach enables us to explore musical territory that might otherwise be inaccessible.

Capturing material


With the emphasis on the hands-on and the physicality of musical gesture, it can be a challenge to capture and recall musical ideas when they present themselves. This may be regarded as a feature of the performer-centric approach by musicians working in the free improvisation space. For example, in my interview with Jeff Kaiser (2021), he describes how his own system does not allow him to save and recall certain configurations: a deliberate choice he made to avoid his improvisations becoming too predictable.

Adrián and I, on the other hand, come from a classical contemporary music background and are interested in producing structured, repeatable, rehearsable works of music. For us, this means not only being able to discover musical ideas through exploration, but also being able to capture, analyze and refine them. Musical notation, our go-to tool as classically-trained musicians, is not well-suited to describe the gestural types of musical material that we are looking to produce, especially when considering the wide range of possible interactions between the saxophone and the computer. Without the ability to rely on notation, we face certain challenges related to documentation and communication.

Our project’s research-based framework and the emphasis placed on documentation, led us to an interesting solution, which we now use as a regular part of our practice to capture and evaluate musical material: recording segments of our working sessions with high-quality audio and video. Additionally, we often include a screen capture from the computer(s) in order to identify the software configuration at any given point. The video at the top of this page is an example of how we record ourselves and document our shared experiences, taken from a session we did while working on the material discovery phase for manifold.

This type of audio-video capture can be a powerful tool for documenting the working process, but we are not simply interested in keeping track of our previous experiments. The ability to review the video immediately after (or during) the session is one of the benefits of this method. The act of watching the video together and commenting on our thought processes is a technique known as stimulated recall (SR), an introspective research methodology (Ryan and Gass 2012) that originated in the field of psychology (Ericsson and Simon 1984) and has since found applications in a variety of other fields, including naturalistic research (Lyle 2003) and music education (Rowe 2009). SR is essentially a structured method of reflection in which you use video review to attempt to verbalize the thought processes associated with your actions (Ryan and Gass 2012). This method can be especially useful in facilitating collaborative work and shared reflection because it encourages us to develop ways of communicating about the music we want to create.

When reflecting on our working sessions, it can be challenging to verbalize thoughts about abstract musical material. We can draw on existing musical terminology such as 'microtonal', 'climax', 'tension/ release', 'agitato', 'misterioso', and so on. We can also use saxophone related terminology, such as 'aggressive multiphonics', 'frantic key clicks', or 'work up to a high D'. There is also a lexicon associated with electronic music, such as ‘modulation rate’ and ‘wet or dry signals’, as well as terminology drawn from or inspired by the history of electronic music, such as ‘objet sonore’ or a sound described as ‘Stockhausen-esque’. These terms provide a broad framework for discussing specific musical concepts, but they often serve only to describe music on the surface. In contrast, we have discovered that much of our creative decision-making occurs at a deeper, more abstract level, and that standard musical language is insufficient to describe the thought processes that underpin it.

Forsythe had assembled a language which later he called Improvisation Technologies. It was an ample alphabet of isometries or transformations that could be applied by a dancer to movements and postures. ... Some of the terms were just versions of geometric transforms: translation, dilation, contraction and rotation, others were specific to dance practice, like 'avoidance' or 'floor reorientation.' The alphabet consists of embodied mnemonics that come from many frameworks of dance and choreography enabling agents who, though they might not be equipped to speak about geometry, were all masters of mobility in three dimensions.

When we talk about [our music], speaking in metaphors of what it sounds like, descriptions as close to the sound as possible, or maybe words that become placeholders for bigger musical gestures.

Such methods allow direct manipulation of the composers’ model in a loop, with the composers’ ears in the middle. The time it takes to cycle this loop is a critical part of the discovery process. If a particular path seems difficult, rapid feedback on one's hypothesis can make the difference between attempting the path or not. If for instance the ratio of interesting to uninteresting discoveries is 1%, a hundred cycles may have to be traversed to find the good bit. If the time to make each change is five minutes, it could take over eight hours to hone in on one answer. If we could reduce the average cycle time to say, ten seconds, it would take twenty minutes to make one hundred experiments, a much more likely time to consider devoting to hunches or even essential refinements in a composition.

The need for 'hands on' in performance forces the composer to confront the abstractness of the computer head on. Each link between performer and computer has to be invented before anything can be played. But these 'handles' are just as useful for the development or discovery of the piece as for the performance itself. In fact the physicality of the performance interface helps give definition to the modeling process itself. The physical relation to a model stimulates the imagination and enables the elaboration of the model using spatial and physical metaphors. The image with which the artist works to realize his or her idea is no longer a phantom; it can be touched, navigated and negotiated with.

Joel Ryan

 ‘Some Remarks on Musical Instrument Design at STEIM’ (1991), p. 5.

Joel Ryan

 ‘Some Remarks on Musical Instrument Design at STEIM’ (1991), p. 12.