The table aims to illustrate and systematize the relationship between musical parameters and their corresponding visual effects, as created by the TouchDesigner program. It is important to note that the specific correlations between musical elements and visual outcomes were intentionally selected by Damiano and me. Within one experimental cycle, we chose to analyze the musical elements using the same program settings for consistency in our documentation:

-      The change in dynamics is correlated with the particles' speed of movement, meaning the louder the perceived sound, the faster the particles move through the space. With a quiet sound, the particle cloud moves more slowly. Additionally, there is a correlation with the birth of these particles: the louder the perceived sound, the more particles are generated, whereas only a few are generated with quiet sounds.

-      The intensity of the sound (for example: vibrato or non vibrato, air support) influences both the speed of the particles and the intensity of the turbulence. The turbulence is an external force that moves the particles through space, in addition to their own random motion. In the program, this turbulence is also known as wind, which clearly demonstrates the function of this force.

-      The frequencies, so the three different categories controlled by the audio analyzer, have different effects on the process. The high frequencies, or the high register of the saxophone, lead to a higher birth of particles, resulting in a higher number of particles in the space. The middle register, like the changes in dynamics, affects the speed of the particles, while the low register influences the intensity of the turbulence, also known as wind.

This chapter aims to present the approach to the software TouchDesigner in more detail and highlights the thought processes involved in the experimentation phase.

Developing this type of interaction involved ongoing experimentation and evolution at multiple levels to discover the most effective approach. The artist Damiano and I worked intensively with the program TouchDesigner to transform the various sounds of the saxophone into visual live impressions. The first artistic, interactive “scene” that we created is called particles cloud, consisting of many small dots/particles that change depending on how the saxophone sounds. The processes in the program are technically complex, but at their core, they involve the following: a sound or noise is captured by the input source (in this case, the computer), then processed by the program, which in this specific scene, generates a certain number of points or particles. These particles have a limited lifespan and will fade or disappear when there is no new auditory input.

One of the fundamental elements of our experimentation was to calibrate the program to the frequencies of the saxophone in order to make the interaction as specific to this instrument as possible. This required hours of testing and programming. The key observation in this regard is that every sound or noise registered by the input source is scanned by the program using an audio analyzer. This analyzer is divided into three categories: high, mid, and low. We therefore tried to adjust the properties of this audio analyzer as closely as possible to match the actual high, mid, and low registers of the saxophone. Through this specific setting and adjustment of the program's sensitivity, we were able to improve the program’s response, and consequently the particle cloud, to the various saxophone sounds. It is worth noting, however, that the saxophone always produces many overtones that contribute to its distinctive sound. These overtones are still partly captured by the program, meaning that especially with the lower notes, the mid and high frequencies often react as well.

To give our experimentation and artistic exploration a more scientific approach, we consistently recorded our sessions and later analyzed them, documented the various experiments, and reflected on the outcomes and findings together. This led to a drive to document as precisely as possible which sounds and sound qualities correlated with which effects in the program. To do this, we listed various musical parameters and systematically examined them to define the corresponding consequences and reactions from the program. The table shown below is the result of this process:

The next two videos presented here showcase two distinct audiovisual scenes developed in collaboration with the visual artist Damiano: particles cloud and moon phases. Both scenes demonstrate the potential of interactive visuals generated in real-time through TouchDesigner, responding dynamically to the acoustic characteristics of the saxophone. In both cases, the musical input is the pre-recorded saxophone part (without piano accompaniment) of the introduction to the first movement of Fernande Decruck’s Sonate en Ut#, recorded by me. This excerpt was chosen as it covers different registers of the saxophone, incorporating both lyrical passages and rapid, fluid technical sections. These variations allow for a rich exploration of how the visuals react to changes in articulation, dynamics, and frequency range.

The particles cloud scene consists of countless small dots/particles that emerge and evolve based on the saxophone’s frequencies. The movement, density, and lifespan of these particles are directly linked to the musical input. Lower frequencies generate a different spatial effect compared to higher ones, and as the saxophone moves through its dynamic range, the particles adapt accordingly. As previously discussed, a key challenge in this experiment was calibrating the system to respond specifically to the saxophone’s frequencies.

The moon phases scene, in contrast, presents a structured yet fluid visual narrative, inspired by the cyclical transformation of the moon. Here, the visuals change shape, surface structure and intensity in relation to the saxophone’s expressive nuances. This scene explores a different kind of interaction, focusing on organic transitions and gradual transformations rather than discrete particle reactions.

These two examples represent potential final products as they might be presented in a live performance. However, it is crucial to highlight that, in this recorded format, the full exchange between the auditory and visual elements is not entirely realized. In a live setting, I as a musician, would adapt in real time to the evolving visuals, creating a more fluid and immediate dialogue between sound and image.

Additionally, the colours of the particles in the particles cloud are adjustable and can be customized according to artistic preference. A future development that Damiano and I aim to explore involves integrating changes in particle colours directly with variations in saxophone timbre. This would open new possibilities for deeper sensory interactions, further enhancing the immersive experience of the performance.

note regarding the visuals:
the birth of the strands depends on the birth and lifespan of the particles,
this means: the more particles are generated, the higher the probability that such strands will form, which, in turn, happens because they are created when there is a high concentration of overlapping particles