The limits of the relationship between human and machine has always become an interest to me as a musician. At some point of my workflow, I deal consistently with the question of how necessary it is to be dependent on such a large amount of high technology and what is the point of defending a workflow where you don’t share its challenges, tasks and pleasure with other humans. During my music education, the most common automotive systems seemed to be designed for only one purpose, a special instance or a specific music instrument. Besides it, the precision and binary thinking of its mechanisms seemed to always arrest our creative flow instead of releasing it.
In our current world, the machine culture has matured to the point that craftsmen’s ideas seem to be some sort of mediator between the analogue and digital world. When those tools, features or products are produced by international marketing and industrial companies, some can also showcase some redundancy of necessity and seemed only to be existing to prove the abundance of wealth and power from developed countries. I have been looking for an alternative in this cultural pattern where we could develop new relationships between humans and things, bringing awareness to its practice and a constructive way of redefining roles and labors, such as by creating new relationships with an instrument or the instrument by itself.
2.1.3.1. Context
Since my bachelor in composition (2011-2015) I have been living in a context where the application of technology is mostly a solution to achieve fluency and dynamism in electroacoustic performances. I have chosen to explore in my own practice the development of an instrument-inspired gestural controller by realizing that my main interest in developing the technical part of Knurl was to find strategies to allow interaction, customization and dynamic changes to be presented in gestures with sound movement and layers. Inspired by the artistic vision that Yampolschi Once shared in this thesis, “every voice or instrument in a music composition should behave as if it represents a character of a story, having its own rules and qualities to share into a global scenario” (YAMPOLSCHI, 2011).
These thoughts gradually became compositions in the course of years. I gradually started to add automotive mechanisms into those creative practices in order to help the administration of each layer. This whole testing process was a work and reflections for years of more tests and trials. By dealing with the craftship of machines for a long time, I started to visualize my own context collecting some challenging practices that when using digital technologies, had influenced and framed the organization of most composition processes:
- CAD thinking- erasing the scratch experience
This issue was presented in the book ‘The Craftsmen’ and it made me realize its presence in my work. CAD is a software commonly used by designers, architects and engineers to design their project. Its manipulative, educative and well presented platform offers to the user a great workspace to use for such complex projects. However, its preview of visuals blurs our experience of sketching. Sometimes, the eyes can’t tell you all the aspects that you need to know from your design. Sometimes, feeling, sensing, listening is also part of a scratch experience. As an example in the book, the author mentioned that architects should spend more time in the places that their buildings will be made to understand its space, in order to understand its sunlight direction and air ventilation, instead of spending months looking into a preview of the computer’s screen.
- The rabbit hole - experiencing deep research
My cello teacher (René van Munster) used to tell me to take care of going down to this rabbit hole when exploring too deep about a specific topic. This behaviour can be defined when precision and binary thinking retain a flow into a creative process by its complexity. Instead of allowing us to test and practice hypotheses, it brings us to a long and trivial journey, where answers become to be made without answers.
I noticed , for example, that most of the goals I wanted to achieve didn't even need to use technology to achieve certain results. Most of the time I used to visualize impressions, feelings, concepts and experiences and I didn´t need to apply and understand complex structures to prove any value to research questions. Going too deep into a specific topic for a small cause can easily arrest your intentions, making the researcher witness of procedures, techniques of perfectionism.
- (Re)evolution - Becoming a trend
‘Machines aren't a fast way of making things, it is slower than our hands’ (Robert van der Broek, 2021). Feeling tempted by the magic of technology is a common trap that usually distracts the researcher from achieving its goals just because there is a large interest and stimulus for this kind of approach from current external influences. This feeling isn't only shared between me and Rob, I noticed that same concerns are shared with collaborators to work with me when I am applying/using technology. It is about time to think about if it is a decision coming from your own intuition or if it is stimulated by trends. Because in theory, any machine won’t make any process faster, more effective, cheaper or reliable, that is always up to your design concept.
2.1.3.2. Goals
To design an automotive music system that allows the control, maintenance and assistance of independent music layers into music performance. It would become a feature for Knurl that allows the performer to receive advice and suggestions from a machine system in order to trigger and manage music events from the instrument electroacoustic interface.
2.1.3.3. Starting point
Machines are a product of our way of thinking. It can be seen as a third arm prosthesis but it also frames us to think in a specific making procedure. With the intention of making a clear and an open-frame starting point, I started some pilot testing involving either a digital or an analogue system that could allow the instrument player to manage their musical ideas using different rules or processes of composition. I have chosen to explore the combination of strings (Analogue prototype) and an advice tracking system (digital prototype):
Analogue prototype:
I have been fascinated with instruments using rotational mechanisms for the production of the sounds: how does our hands redefine its role from the craft into leadership and what are its expressive characteristics that this combination could offer into music composition. My startpoint of this journey was my relationship with the bow.
After experimenting with the 2 bows technique from Frances Marie Uitti, I became fascinated to explore a way where I could play multiple strings with the same expression and variations that a bow could offer. I started to explore ebows, motors, hurdy-gurdy systems, motorized bows but none of those replace the amount of accents and variations that a simple bow could produce. I invited the architect and instrument builder, Mihalis Shammas to collaborate on this question with me.
The bow design started to get after a longer evaluation until the end of this master thesis. I reflected for a long time what it works and what I miss from a traditional bow in my music practice. This prototype demands further investigation before I share its key findings. After the conclusion of Knurl's general shape I hope to be able to share the findings of its application.
Digital prototype:
At the same time, I was also wondering if the physicality of machines and its mechanism would completely achieve my research goals knowing that any physical action/space already contains its own rules and complexity. Approarding this research with a digital format would offer more flexibility to organize and filter a system and its rules. This made me consider doing some experiments with artificial intelligence too, using a programmed system where Knurl could interfere into the music development by generating multiple advice along its performance. For this prototype, I created a research group composed by Farzaneh (Composer, programmer and cellist), Yota Morimoto (Composer and programmer) and Timo Hoogland (Sound designer, programmer and drum player) to explore this other face present in our current discussions about machine and music.
2.1.3.4. Application
Our group collaboration for the machine learning design was a durable and gradual experience over months. Our first goal was to explore the power of giving voice to the matter, the instrument. We wanted to create a sensation for the musician to be playing in collaboration with someone by playing knurl. Not in a way that excludes the possibility to play with real people, but as if you are playing with a judge trying to summarize the musical paths you are choosing and orienting you through sounds.
In the summer, The group has decided to work with MarkovChain defining probabilities for specific scenarios of 4 different kinds of data: Frequency, amplitude, ZeroCrossings and Flatness. However, after reflecting about this approach, this wouldn’t make a mechanism learn or listen itself, it would only give me uncertainty of its possibilities. I decided to test myself a system in which an advice could be calculated and a suggestion could manipulate the form of the music and therefore, aiming for a balance to the structure of the music.
The system is designed based on 2 concepts of music that I used to have in mind as a composer. One of them was collected in a reading of the book ‘Meta hodos + Meta Hedos’. The idea collected from the reading of this book is based on the fact that the longer and the further we listen to music or a sound, the more used to it we get to the sound and therefore, more time we need to listen to its event, understand, organize it and associate with the previous information. Tenney also mentioned that the degree of intensity can also influence this decay of attention, for example, if you play a percussive note for a second time louder and equally, this note won’t surprise you if you won’t prolong the silence time in between if it is short. Those thoughts can seem simple to understand by your own experiences but when you are composing complex structures, it can be a nice reminder and a simple general rule to have in mind.
Another concept that this system is based on is an own interpretation of sound material in a degree scale. This idea came from improvisation practices of my cello and in the Music and Dementia classes with my teacher René van Munster. Once Rene mentioned to me that his previous teacher used the metaphor of ‘the mirror and the hammer’: a figurative idea of how a music material can be understood as a repetition (mirror) or development (hammer). With this in mind, I started to imagine a system, in which If a musical idea is presented it can be understood either as a repetition, variation or development. This is represented in a scale between 0 and 1 (illustrated below) and used as the basis of understanding to generate and give advice for the musician. As an example, If the musician is playing for a long time with only repetitive material, the system would trigger sound material that engages contrast and stimulate the musician to work for a development.
|
Repetition |
Variation |
Development |
|||
|
0 |
1 |
||||
The longer the music is, the less often those analysis and advice are triggered. Some mechanisms designed at the digital buttons of one of the modes of performances (detection mode) can, although, manipulate those trigger frequencies allowing the musician to have a certain control . This is because those concepts can’t be followed literally in all music contexts and scenarios. The analysis stores only data from the beginning when the system is booted and in a music concert, multiple factors can also determine the generation of the material (program selection, duration of the whole concert, audience, etc.).
The system was tested with a composition called Idalina, therefore, most of the sound material organized to be triggered had influence from this artwork. The sound material selected for this prototype was a short event of electronic or live-synthesis sounds, in order to better perceive the reaction of the machine in combination with the musician’s performance.
2.1.3.5. Findings
For the music compositions using the mechanism of Machine learning, a list of events simulating patterns of repetition, variation and development had to be previously defined. At this moment it is possible to not only trigger special sound material but also trigger it with variations defined by a calculation of an advice.
However, as mentioned previously, music material has a lot of parameters related to its production and it is almost impossible to diagnose without a small human guidance/manipulation. In the future it would be nice to construct sounds also by machine generation systems but at the same time, I wonder if the musician would feel safe enough by relying on the machine to compose, since certain musical ideas do not necessarily follow rules strictly.
Regarding the analogue prototype, the bow design started to get after a longer evaluation until the end of this master thesis. I reflected for a long time what it works and what I miss from a traditional bow in my music practice. This prototype demands further investigation before i share its key findings, however I will share below some pictures of its current state.