New projects like the Data Sonification Archive 1, are an attempt to bring together the exponential amount of artworks being created in the field of sonification. In this archive, one can find podcasts like "Loud Numbers" (2021) 2- the first sonification podcast in the world, to artworks like "A Galaxy of Suns" (2016) 3 - a performative interpretation of stars' positions.
Generally speaking, sonification projects are usually inspired by topics that deeply concern humanity and their goal is to show another way of understanding, but, as questioned by Dr. PerMagnus Lindborg, “can we gain a deeper understanding [of science] by listening to a sonification of the data that scientists use?” 4.
While the idea of understanding science can be very broad to define through sound, what in fact sonifications aim for is accessibility. In this line, Dr. Palle Dahlstedt expresses his thoughts in his chapter Evolution in Creative Sound Design 5, stressing the need for research to focus on accessibility and a more intuitive synthesis process. This research could be applied to how sonifications interact with the users, creating interactive human-machine systems to reinforce access to information. We can use as a case study Lindborg's Leçons 6 research paper on interactive systems. In his paper, Lindbog creates and analyzes an interactive performance system, where the sound produced by a performer is analyzed by a machine learning algorithm, creating a live-music response. The result is a co-creation between a human and a machine.
This idea of co-creation is a strong feature when combined with data from our environment. Sonifications have the potential to become responsive and interactive when combined with AI techniques and sensors. So, how do sounds and other materialities come together in a multimodal representation to convey the inherent meaning in the data?
The sonification artist and researcher Sara Lenzi 7 states: “we see the use of sound together with other sensory modalities as a step forward toward the engagement of the public with technoscientific knowledge such as that surrounding climate change”, where sensorial devices are used to reinforce a scientific message. In addition, Moritz Stefaner 8 has been doing notorious work in data visualization, working with the limits between data visualization and data art, especially in his recent investigations using the AI MidJourney.
There is still a great potential in the exploration of interactive and intelligent sound design, in relation to data sonification. And that is the way that Sounding Numbers will follow after this period inside RMC.
These investigations could potentially focus on the embodied perception of the data, working around an interactive or physical sounding experience, where the focus is set towards participation and engagement through the emotional qualities of positive and critical aesthetic experiences.
A state-of-the-art resource to carry out this future project is Flucoma 9, an open-source project where a toolset of machine learning algorithms is made available for sound processing. At the moment, they have integrated into SuperCollider, algorithms that go from clustering and classification to neural networks, like Perceptron.
I often visit online open-source data projects, to choose research on climate change. These data projects are supporting pillars for community empowerment, and are very easy to find. They also offer reliable sources of information with clean data sets and give their resources free of charge to facilitate work on topics such as plastic waste.
These projects often take the form of organizations and rely on information spreading through various media to achieve their goal of knowledge dissemination. To increase this dissemination, they often promote and support projects aligned with their beliefs.
In the last years, the world experienced an increasing amount of projects and actions in relation to climate change. More projects in the field of activism and artivism are being born, partially due to engagement and awareness being raised in new generations.
This involvement can be clearly understood through Dr. Anai Mauruschat's post Ph.D. project "Sounding Crisis", developed at Sound Studies Lab in Copenhagen 10. In this project, Mauruschat investigates on sound practices of climate activists, the sonifications of climate change by artists, and the environmental practices of Indigenous people.
Last March, during an open panel with the researcher, Mauruschat made an observation based on her investigations, that most of the participants in climate change demonstrations are people between 15-30 years old. She especially noticed, through qualitative interviews, the frustration, depression, or anger of the participants, as an emotional state coexisting with strength and confidence in fighting for change.
In my work, I try to express the urgency and fighting spirit of the era we live in, but I try to address it from a positive standpoint. We need to take action, but there is enough activist and artworks pointing toward the anger and depression of immediacy. In my creations, I try to take the other side of this fight.
A clear example of the strength of people working together for climate justice can be seen in “The Montreal Protocol”, where the world came together to ban and decrease the use of CFCs to stop the decay of the ozone layer.
a short video about The Montreal Protocol here
The main topic of my project can be defined as ways of exploring sonifications as artistic materialities through code for engagement and interactiveness. It can also be understood under the concept of SciArt, a term that can be traced back to the 1960s with the exposition of the piece Homage to New York at the MOMA 11 and can be defined as the following: ‘The post-disciplinary practice merging elements typically associated with The Arts and The Sciences which leads to exploration in those areas’ 12. Since the '60s, we can trace a rising tendency in institutions like the CERN, SciArt initiative, Cosmos program (IRCAM), Chalmers University, the Honk Kong Climate Observatory, the observatory in the Atacama Desert (Chile), and Campus Biotech, among others.
These institutions create the possibility for artists to engage with scientists, offering access to scientific methods, data, and technological resources. The goal is to find new ways of interaction based on cross-disciplinary encounters and find new ways for scientific dissemination through art creation. The programs often focus on how to work with community engagement and interactiveness through technology, and new ways of exploring scientific instruments as musical instruments.
The first step in my creative process was to choose a topic that resonates with my beliefs, and from which I could create a new poetic perception of reality using its theories and data relations. For this project, l worked with neutron stars, deforestation, and ocean pollution, which are part of cosmology and Earth sciences. I find that these two fields carry a beautiful feeling of wonder and empowerment, and are great with open-source data.
While reading about these scientific topics such as deforestation, I encountered vast amounts of information, so my first filter was to focus only on scientific dissemination channels and research papers. In order to filter the data sets, I only used open-source CSV files offered by the NGOs or governmental institutions that supported the research I read.
I conducted my artistic practice-based research project Sounding Numbers, as part of the Advanced Postgraduate Diploma at the Rythmic Music Conservatory, from September 2020 until June 2022.
During this time, I researched how to create sounds with scientific data and how to use these sounds for artistic outcomes. I used programming techniques to find structures and relations between the data, the scientific theories, and sound potentialities, where I explored textures, qualities, different data mappings, and ways for sound deployment, development, and cessation in a musical context. In brief, I investigated how can I use different data sets to craft synergies between science and my aesthetic music universe, under the research question:
When I wrote my research proposal, my coding skills were on a very basic level, so naturally, diving into a research project involving so much work in this area was risky, but at the same time exciting! So I asked myself:
I decided to trust my experience and professional skills in composition and performance, so I presented Sounding Numbers. One of my goals was to develop structural and creative programming tools in the context of my artistic practice. I believed that going through this process in a research context, could guide me to new aesthetic parameters and potentialities within programmed sound designs, and help me find a more defined way of blending acoustic and electronic instruments within my artwork.
In this sense, planning was a key point for success, so for the four semesters that I had ahead, I decided to explore different coding languages with increasing difficulty, and apply that knowledge to my regular practice toward an artistic outcome.
In an overall sense, I think that researching new and unknown materialities, in the frame of an artistic practice-based research project, can create a strong setting for novelty discoveries, opening a potential for knowledge and spiritual artistic development, if the planning is set to realistic but still ambitious goals.
The following step was to understand the theory and data. My goal was to create an emotional connection with the material that I decided to work with. Consequently, while reading the theory and data, I looked for potential sound relations between the concepts and my musical universe. Thus, to understand the data, I used data mining analysis techniques, among them clustering, plotting, and predictive algorithms to discover relations and tendencies hidden in the numbers.
These analyses and understandings sparked my imagination with relations between numbers and musical characteristics, e.g. dynamics, shapes, and overtone developments. In my project, the scientific theory was related to musical structures, like the Niche Theory. This theory focuses on the singing range of species, which I used to create linear relations between orchestration and passage developments, by establishing cross-references between instruments.
How can I use scientific data sets to create expressive sounding tools
´
for contemporary composition and participatory sound installations?
For these pieces, the sonifications were recorded as samples and used as an expressive toolset by the electronic performer, who operated them via MIDI controllers. In addition, the acoustic instruments were written using these sonifications as a source of inspiration for their melodic and harmonic shape, textural correlation, pitch set, and extended techniques development.
On a composition-design level, the blending process between the acoustic and electronic was designed to nurture each other. When writing the code for the electronic sounds, I had in mind, which textures could I relate to, and how the sound could be shaped, so it could easily fit different extended techniques. At the same time, while composing the music, I knew beforehand which textures, shapes, and movements the electronic sounds could develop for specific extended techniques.
In his lecture at Harvard University 13, Pierluigi Billone deals with the relationship between sound practice, sound thought, and our technologically transformed reality. In his notes, he states that 'sound conception' and 'sound culture' are always defined and active before we hear a sound, and they are always contained in a revealing relationship: “a reveling relationship between me and it [sound]”.
My position on not using field recordings is supported by Billone's reflection. I allowed the electronic sounds to create a perception of nature through the imaginary and emotional means of the listener. I facilitate through this action a "revealing relationship" between the listener and the sounds I propose, reinforced by not using natural sounds, that contain a linear relationship to our life experience. The revelation I propose needs an effort in the act of listening, and it is with this effort that a relationship is discovered, rooted in emotionality and vivid imagination.
Billone also reflects in his lecture, that sound contains a balanced identity, and when this identity is collapsed, it becomes noise. He believes that the identity of sound can only be created using imagination, an ability that a computer does not possess. In my music, I create a sense of balance by using melodies that resemble a folk tradition. I believe that with this action, the electronic sounds that I create with a computer, reveal a potential cultural belonging. Therefore, facilitating an emotional association.
I did not have yet the chance to meet with danish composer Niels Rønsholdt, but his interviews and artworks inspired me to seek the emotional engagement of the performers in my music, through a variety of techniques.
Niels Rønsholdt explains that his work aims to reduce the distance between artwork, creator, and audience. One of the ways Rønsholdt fulfills this goal is to put himself into performative roles - even though a dedicated performer would have a more prepared set of skills. By doing this, the artwork, creator, and audience are able to share the same performative space, creating proximity in the experience.
Rønsholdt also reduces the distance between performer and creator by meeting with instrumentalists before composing. By doing this, he explores musical possibilities from the performer's toolset and instrument, which then he evolves into abstract sonorities to compose with.
His methodologies open a space for reflection in my work as a composer. My musical language has always been complex, but I strive for accessibility. In this sense, I try to reduce the distance between the performer and the composed material by writing demanding passages, and exhausting the material through the act of playful struggles. What I aim for in this struggle is to create a sense of familiarity, as the performer needs to give their full attention to solving specific music passages. Once these are resolved, an emotional connection is created between the music material and the performer, shortening the distance.
Portrait of Niels Rønsholdt here
Overall this music falls into thespell of complex sonic textures, that are broken by emotional melodic qualities.
The compositions are the most abstract layer in the project, and the scoring technique played a fundamental role. The music was written with a mix of detailed-notation that requires a solid technical ability, and graphic-notation
My work on detailed notation also references the work of Danish composer Jexper Holmen. In 2019, I took lessons with him to work on detailed composing techniques over a small set of parameters. We discussed over these meetings, how to create a clear overview of the materialities, and how to develop long segments allowing micro-nuances of the sound to be amplified by perception.
Holmen's concept of detailed notation can be seen in my scores, where the music goes through structural repetitions with small probabilistic variations over a limited set of materialities. For example in sections where high-pitched harmonics with long-glissandos, are played with different bowing techniques throughout the repetition of the material.
Portrait of Jexper Holmen here
Notes on Pierluigi Billone.
One of the pieces that informed my composition process for Sounding Numbers is ‘Face’ by Billone (2016) 14.
Billone uses contrasting atmospheres, that one can hear lurking and raising over each other, nevertheless coexisting in the same environment. In his music, non-pitched loud, short, and sharp sounds, coexist with low, sustained, muffled, and atmospheric textures, as heard in the movement Face IV from 8’10’’ until the beginning of Face V.
Throughout the movements, the saxophone plays multiphonics creating resonating membranes, which are disrupted by short and soft melodies. Both elements melt into each other using as buffers mezzo-piano crescendos, allowing space for alien gestures from the performers.
I used this idea of opposing and coexisting materialities in both of my electroacoustic pieces. These contrasts can be heard in the way melodic movements arise from complex textural soundscapes to develop their sonority and move the music forward.
Klaus Lang, unknown date. “the breathing of time” 15.
“When exploring the very heart of the instrumental sound, by looking at it through a magnifying glass, the already known and everyday sounds become again unknown and wild territories.”
Under this concept, I explored detailed notation in my compositions. By looking through a magnifying glass at my music materialities in combination with extended techniques, I created an interesting layer to investigate.
For example, applying different levels of pressure on the bow in combination with written harmonic glissandos became a sound territory of controlled frequencies with wild intervallic and textural relations. The multiphonics on the saxophone became a layered structure where I could apply different dynamics and independent vibratos.
It was interesting to discover that in the act of performing such complex sonorities, our listening experience as performers intensifies. We, as performers, fall into the timeless spell of textural development. As Lang reflects on his essay: “Through listening, time becomes eternity”.
Why not field recordings? When we hear recordings of natural environments, our memory connects with them. We perceive them from the outside and enter their space by listening. In contrast, I believe that by using an electronic abstraction of natural environments, we recreate them by perceiving ourselves. We enter our own space, our own imagination, and we connect the new electronic sounds through different memories of what they could represent to us. The effort that this requires, awards us with an increased sense of sensibility.
In both types of artistic outcomes, I merged the electronic and the acoustic, and the conventional with the extended
Maki Solomos’ “notes on Francisco López” 16, gives a great insight into Lopez's artistic methods and intentions in field recordings. Solomos analyzes López’s works as composed field recordings and states that in his works there is a truth of wilderness and chaotic nature, that one can only find in composers like Xenakis.
In my sound designs, I tried to create a similar wilderness that Solomos refers to. I aimed to generate an electronic universe that could convey the genesis of nature. However, I did not seek the mimics of a natural soundscape, but I looked for an electronic abstraction that could potentially find an emotional narrative by being wild and ‘noisefull’ in correlation.
The emotional narrative of the sonifications is based on their numerical genesis. The numbers show a human relation to them, which can easily evoke an emotional reaction. A clear example can be heard in the sonification artwork "Outro Registros" by Brazilian artists Espinoza, Rafucko, Van Ransbeeck, and Holmes 17. In their work, they denounce police crimes in favelas. The impactful installation uses a sonification based on deaths by police gunshots, that in the context of the artwork brings a vivid layer of emotionality.
Field recording and sonifications can become thrilling sounding experiences. While on field recordings, one steps into a sound milieu to observe a natural event that has already happened, sonifications ask the audience to create a sounding environment through a combination of electronically generated stimuli and contextual relations. These sounds have the potential to use historical data or predict events in the future.
An interesting field to develop would be the combination of both, to investigate how data sets through algorithms, can interact with field recordings. The combination could potentially open a new interactive form of expression for audience engagement and awareness.
I composed this music for alto and tenor saxophone, cello, double bass, voice, and electronics. The orchestrations served my goal of working with multiphonics and various attacks on the wind instruments, harmonics and different bowing techniques on the strings, and extended techniques on the voice.
The performers I chose for this project have professional backgrounds in classical or rhythmic music, all of them with experience in improvised and scored music. I think that working together to perform compositions that address such heavy topics -as deforestation and ocean contamination, brought from the deepest of their intentions, a feeling of compassion and cooperation. The virtues of both backgrounds came together in the recordings to enlighten the compositions’ requirements with the highest of the ensemble’s capacities. A common goal was achieved within both worlds, something that does not happen so often between rhythmic and classic players.