Insights through spatial visualization of the compositional decisions regarding auditory perception in a hemispherical loudspeaker array

Jakob Gille

This research explores the crucial role of spatialisation within electroacoustic composition through a detailed analysis of a section of the author's work 'Motion', composed in 5th order Ambisonics for hemispheric loudspeaker arrays. It focuses in particular on how spatial analysis tools can reveal perceptual relationships that traditional spectral analysis alone cannot adequately capture. While spectral analysis provides invaluable insight into timbre and frequency content, it cannot show the dynamic interplay of sound sources in a three-dimensional environment. As spatialisation profoundly influences how listeners perceive individual sound events, their interactions and the overall coherence of a composition within a given acoustic space, the analysis of it is even more important. The author also draws on the importance of analysing spatialisation to provide insights that are otherwise not easily understood. One of the many challenges faced by a composer working with multichannel audio is the uncertainty of the actual loudspeaker configuration of the concert space. A spatial analysis of immersive electroacoustic works can provide insights into how these problems can be addressed through compositional decisions based on auditory perception. Such analysis can help composers to anticipate potential problems arising from different room acoustics or listener positions, ultimately leading to a more consistent and effective listening experience in different venues. Linked to this, the research provides an overview of auditory perception within hemispheric reproduction systems, building on Kendall's framework that "listeners experience electroacoustic music as full of meaning and significance, and they experience spatiality as one of the factors contributing to its meaningfulness". Drawing on Bregman's auditory scene analysis, specifically stream segregation, the analysis thus focuses on how spatial techniques can address masking effects through spatial separation, as discussed also in Brümmer's work on the decomposition of sound material through spatialisation. By combining multiple analytical perspectives in the form of spectral analyses, contextualised with spatial analysis tools, this research aims to provide new insights into the relationship between spatial composition and auditory perception in contemporary electroacoustic composition, contributing to the growing body of knowledge on spatial perception and compositional practice in immersive audio environments.

First Experiments

In "Spectromorphology: explaining sound-shapes" Denis Smalley draws attention to the problems that could arise when working with scores and for the analysis of electroacoustic compositions the use of sonograms: "But a sonogram is not a representation of the music as perceived by a human ear — in a sense it is too objective. Its shapes therefore have to be interpreted and reduced to perceptual essentials. In other words, someone has to decide what to retain and discard from the representation, and more particularly, try and determine how much detail is pertinent to the alert listener." [Smalley, 1997].
In my first experience, the importance of this statement became immediately apparent as I became aware of the limitations and possibilities of a spectral analysis in the form of a sonogram, for example. Several measures are taken when creating a sonogram to try to make the visual representation more accurate in terms of auditive perception. For example, using a logarithmic scale instead of a linear scale for the frequency and amplitude axes (using the dB scale) already approximates human perception of frequencies, but other perceptual particularities are not taken into consideration. The sonogram reveals details in the music that I was not previously aware of because they were masked by other elements.
The use of spatial analysis tools makes the problems even more apparent. In my composition 'In Motion', I draw particular attention to auditory perception and its various features. One such feature is that sine tones are notoriously difficult to localise, and only sounds with a greater number of overtones, closer to broadband noise, can be localised accurately. However, spatial analysis cannot demonstrate these differences and instead depicts the source as much sharper than it actually is.
Due to the aforementioned difficulties and challenges, I decided to use a combination of tools to highlight different aspects. In addition, comments with timecodes provide context for the analysis, highlighting its strengths and weaknesses. "There is no objective method of achieving a visual spectromorphological representation, and the analyst hopefully becomes only too aware of subjective decision-making and alternative ‘readings’. This is as it should be." [Smalley, 1997].

In the following, I will present and describe the analysis tools for frequency, amplitude and spatial analysis.
In several experiments, I examined different tools for analysing the spectral and spatial content of my composition. It quickly became apparent that no single tool would be sufficient to demonstrate the intricacies of a spatial composition, given the issues of auditory perception and the often inadequate visual representations of these. I therefore experimented with different combinations, examining their usefulness in showcasing my compositional decisions and their impact on perception.
For the frequency analysis, I first selected the standard waveform analysis available in the DAW Reaper. This displays time on the x-axis and amplitude on the y-axis. Additionally, the waveform is coloured according to the most dominant frequency. Lower frequencies are coloured blue and higher frequencies are coloured yellow, with different colour grades in between. This makes it easy to quickly explore the different sections of a piece, as they are clearly denoted in the waveform analysis.
I'm also adding a standard sonogram, also known as a spectrogram, which displays time on the x-axis and frequency on the y-axis. In this spectral analysis, different colours and their intensities indicate the amplitude or energy of a certain frequency.

Unlike my initial experiments, this analysis included an additional frequency analysis, displaying frequency on the x-axis and amplitude on the y-axis. However, I wanted to show not just the sum of the individual channels of my spatial compositions, but also the frequency components of the individual spatial components.
As my composition 'In Motion' is in 7th-order Ambisonics, the format's underlying maths works with spherical harmonics. This format, which contains spherical harmonics, is called B-format and, in the case of 7th order, contains 64 channels. The first of these is the W-channel, which contains the omnidirectional pressure of the signal. The second shows content on the left-right axis, the third on the up-down axis, and the fourth on the front-back axis. All subsequent spherical harmonics further refine the spatial image (for a more thorough explanation of the Ambisonics format, refer to [Zotter et al., 2019]).
It is now sufficient to examine the first four channels of an Ambisonics B-Format in closer detail, as they already contain most of the spatial information. The aforementioned frequency analysis was configured to show the first spherical harmonics, excluding the first W-channel as it contains no spatial information. This makes it possible to highlight the frequency content of individual directions and draw conclusions about compositional decisions.

When it came to choosing the spatial analysis tools, I decided on two options that illustrate the spatial distribution of sound in a theoretical sphere or sound field at different resolutions. This allows me to highlight different spatial features in the composition, as sometimes an overview is more useful than a very detailed analysis of every little detail. Both the coloured waveform analysis and the spectrogram are built-in features of the DAW Reaper. However, for the spatial analysis tools, I opted for two different plug-ins: in my first experiments I selected from the SPARTA VST plug-in suite the SLDoA plug-in (based on an estimation of the direction of arrival of a sound) but decided for my final analysis to select the EnergyVisualizer from IEM plug-in suite as its visual representation is a bit clearer. The second is a plug-in called Hemisphere from the Blue Ripples Visualizer suite. This tool provides spatial analyses from the top and bottom of the sphere. The bottom view is not particularly insightful, but the top view provides an overall impression of the spatialisation, which I found helpful when evaluating the overall spatial picture.

The challenge for me now was to combine these different tools in a way that clearly shows their connection. To help with this, I added short descriptive text to every point in time that I wanted to draw attention to or analyse further.
Regarding written text to describe music, I would like to share another interesting quote from Smalley: "But we must be cautious about putting too much faith in written representations because writing freezes the experience of temporal flux. It is a device for counteracting the fleeting and selective nature of wayward aural attentiveness and memory during the sounding flow of music." [Smalley, 1997]. This is why I hope that the combination of these tools will provide a more complete picture than any single element could.

Tip

Tip

Jakob Gille - Insights through spatial visualization of the compositional decisions regarding auditory perception in a hemispherical loudspeaker array - 2025