SELF-INTERVIEW
(answers to some questions that I encountered during the process)
Why did you make this work?
I was looking for alternative, spatial, non-linear ways to experience music.
What do you mean by "to experience music"?
In western context we have learned to understand music through specific roles: you are either giver (performer) or receiver (audience). Musical instruments are seen as a tool for the performer, to be learned and skillfully mastered through devoted practice. Interactive sound installations disrupt this dichotomy, there emerges a hybrid format for experiencing music. Here, the performer is also the audience, the instrument is also a sculptural element and a prefabricated piece of music. As the politics of the space changes through participation, it is no longer a space occupied and controlled by the artwork to be passively perceived, it turns into a relational space, where the audience can assert their own agency.
What do you mean by embodied?
When the materiality of our body (the dimensions, the energy of our movements) translates into sounds, then it is a process that engages the whole body, including the brain and the nervous system. It is through the process of interaction that the audience experiences and makes sense of the sounds, and it feeds back to their brain, their nervous system and their movements.
Who is this instrument for?
This work does not ask who you are, it does not classify you according to your skill. It is made for anybody who likes to play with sounds, not depending on their previous experience.
Why does it only react to hands and not the whole body?
This was a deliberate choice, that is related to the way we use hands to make sense of the world. You reach out your hand as a gesture to initiate conversation, as a motivated act of reaching towards. When you start to move your hand or fingers, the sounds start to change according to your movements. I wanted to give the audience the possibility to shape the sounds in real time, and have the agency to start or stop the interaction at any moment. Rather than placing the whole body in the interactive zone, I opted for hand-movements that enable a more subtle and dialogue-like conversation with the sounds.
After watching the actual interaction videos, and analyzing participant feedback, I see that there are ways I could improve the interaction by also adding layers of sound that activate when there is movement (such as a head or a foot), but not recognizable hands.
Similar works have been made before, for example David Rokeby's "Very Nervous System", where is the novelty of this research?
David Rokeby made a similar, pioneering, interactive work "Very Nervous System" in 1986. You can see a video of him performing on the system here. However, I found that there was also research done in 2009 by Lizzie Muller on audience interaction with Rokeby's work. You can find interviews with Rokeby here and videos about audience interaction here. I am a fan of Rokeby's work, but when I looked at the videos of how participants behaved inside the installations, and how they described this experience, it seemed to me that there is still much we don't know about the psychology of audience interaction with such works. I wanted to make a work that would be different in terms of the principles of interaction and where even the shy audience would feel confident to interact in a very short time, without being intimidated by the space or the interface.
Previously, I have also experimented with interactivity in some of my installations, and while watching the audience interact with them, I realized that people behave differently and unexpectedly. While some participate with eagerness and freedom, some are very careful, especially when there are sounds involved and you are not quite sure how they are controlled and also when others are watching or hearing the sounds. In physical touch based interfaces this might be different, but proximity-based interaction makes people somehow more cautious. I want to understand, what is needed to create an environment to make the interaction actually happen in a way that would be meaningful/enjoyable for the audience?
How "Touch" is a sculpture?
From a visual perspective, it might be confusing, what makes it perceivable as a sculpture. There are "lamps", and speakers, and stands for them, and cables, but where is the sculpture?
The work has clear physical dimensions in the space, there are areas that are interactive and areas that are not (like presence of absence that is laid out in the space). These areas are marked by colored light, a visually perceivable body of light that indicates the dimensions of the form. When a participant places their body (their hand) inside it, then the dimensions of their body and the sculpture intersect. We might call it touching.
The touching is marked by sounds. The sounds are born (synthesized) on the spot as a result of this spatial interaction. The sounds are surrounding the participant, indicating the wholeness of the structure, again pointing to the whole installation as one sculptural element.
The perception of the space, the being inside the sound, the fact that one is interacting with "the space", makes it akin to sculpture as an autonomous form. It is also the space-relatedness of the soundscape, the combined agency of the apparatus and the participant combined and contained within this space.
What makes this kind of reasoning a bit weak is the hybrid nature of this work, where new formats (interaction, computations) and traditional formats (music, sculpture) intersect. We lack vocabulary for this kind of structures, hence I call it interactive sonic sculpture, but in a performative context it could also be called a hybrid instrument since it can also be used and perceived as a musical instrument.
“Touch” is an interactive sonic sculpture and a hybrid instrument. It is a space where sounds are activated and shaped by hand movements.
It explores the relationship between sculpture, sound, and movement-based interaction. It examines how listening, movement and interaction shape our perception of space, body, and the imaginary.
Experiment with the audience.
I invited participants (altogether 16), to make an experiment with the installation. The installation was set up in an enclosed dark room (a recording room in a music academy).
First, I briefly explained how the system works. Then I left the room for about 15 minutes, and left the camera running to film their interaction with the sounds.
I did it in order to understand how they interact, how fast the learning process is, what works and what does not work for them in this kind of set-up. Are they able to come to the point where they enjoy this as a musical experience, or will they be struggling with the ambiguity of the interaction?
After this, I conducted a short interview with them to ask questions about the process of interaction.
What I did not anticipate:
People wanted to move around between the sounds (doing the mixing), more than they were interested in playing with one sound for a long time. For those who were shorter than me, it was challenging to reach laps that were further away (as I made the installation according to my own body size).
People who were naturally moving fast, had many moments when they made gestures, but no sound was responding. This was mostly happening because the algorithm did not recognize the hand in the lamp-camera image (motion blur caused by fast movements and low light). This could be solved by using a more sensitive camera module in the lamp-camera.
There were moments, when participants reached their hand very close to the lamp-camera, and the hand-tracking algorithm failed to track their hand as it did not fit the camera image. This can be solved by using a wide-angle camera module in the lamp-camera.
I used a surround-sound type of sound distribution. In group interactions, it was sometimes confusing, because it was hard for the participant to differentiate who was making which sounds, because the sounds were distributed between 5 speakers and the location of the sound in the speakers did not correspond to the location of the lamp-camera. In solo interactions, the participants on the other hand mentioned that they very much enjoyed that the sound was coming from all directions and surrounded them.
On the positive side, the learning process was relatively fast, though the interaction patterns were somewhat different. Some preferred to explore the space from floor up to discover different sound modalities that were there, while others preferred to do the mixing, starting to play it as an instrument or orchestra.
As the majority of the participants had some kind of experience with playing a musical instrument (since the experiment took place in a music academy), there might be a bias towards participants who are used to performing an instrument . But there were also people who did not have any relation to music, and their learning process was also fast.
Based on the video and interview data combined, it can be concluded that people who took the approach of intuitive exploring, had fewer interaction failures (e.g. hand moving, but no sonic response), than those who were trying to control it too much, as like playing a traditional instrument. It might be said that no experience with playing an instrument might even be an advantage.
One success factor of the interaction was that they had a private space and time, where they were not interrupted and could concentrate just on their interaction. This situation could be very much different in for example a gallery where there are many visitors.
The fact that the room was dark, and the interactive area clearly visible on their hands, added to the experience in terms of concentrating on the sounds rather than the visual elements of the space. Also, there were no external sounds coming in.
From emotional perspective
When I later watched the interaction videos, it felt like watching them somehow transformed my thinking about music. It happened because I watched them interact with the sounds in a very physical and somehow sincere way. What came out was their inner being. I felt like I was there with them. The emotion that was in these movements, the beauty of their hands coming into the circle of light and flowing in the darkness. For me, there was something very real and sincere and beautiful in these movements and in these moments.
There is nothing that is directing them, except perhaps the inner rhythm. The freedom, like thinking through the sounds, the entire process visible, their faces relaxed, smiling…
"We are the proposers. We are the mold. It is up to you to breathe the meaning of our existence into it. We are the proposers. Our proposition is that of the dialogue. Alone, we do not exist. We are at your mercy. We are the proposers. We have buried the work of art as such, and we call upon you so that thought may survive through your action. We are the proposers. We do not propose you with either the past or the or the future, but the now. "
Lygia Clark, Livro Obra (self-published interactive book, text written in 1967, published 1983)
How was the interaction for you?
I realized I went first, very systematically, like going first from one lamp to the other. I think, maybe like a bee, you know, like going from flower to flower and, like, what is this? And what is this?
Did you get this feeling of being in a garden?
Yeah ... or maybe like ... I got very Harry Potter vibes also, like this kind of magic,kind of, and then this kind of how I think it's, if you think about dancing, and you're kind of like forming your body according to music, and then you think about like playing an instrument, for example, like a guitar, when the strings are vibrating, so you're kind of combining them into both here.
hmm ...
Yeah, like, I have seen these people on this, you know, this tactile ...
Theremin?
Yeah, this electrical thing. And I think there's something very maybe like, if I would play an instrument, it would be lovely to play something like with the body, kind of, more like this. So it felt really somehow that you could, I could have continued for many hours, probably like in that sense.
But did you kind of ... Did these sounds kind of evoke some kind of imagination, or this whole experience, did it bring you to some other space or imaginary place?
yeah! Like, I think, like these different places had, like, these different characteristics or characters. And, umm .. like, I think I'm a bit biased, but I really like this one (points to one lamp) right now, or I have listened to a lot of like, organ pieces, so I enjoyed especially the organ or I was, like ... (humms a sound). Especially in November ... this kind of, yeah... but then, I don't know if I got like, it was maybe more about like how they first it was more like figuring out their colors, or a bit like how they talk, and then it was interesting to do both, you know, do at the same time, or or kind of get them to sing together, or something like that. So, yeah, yeah, but somehow, yeah, I somehow enjoyed this (points to another lamp) with the hand, and I tried this, like opening my fingers and closing, and then I realized, if I turn like this, it will shut up. (laughs) Then I realized like you can change the tone like by being like this (makes gesture), and then you switch like this (makes gesture), and then you go up like this (makes gesture), so you have like this.... You can cut in between instead of doing like this (makes gesture). So it was quite interesting, but it was just in the end...
interaction wise, how long did you think it took you to, kind of, like, learn to familiarize yourself what's going on, or was it, was it somehow intuitive or more frustrating? How would you explain that part?
I think I'm quite intuitive person. So for me, it was quite easy to just, like, try, because I feel somehow, because I'm not, like, musically educated or anything. So knowing about the scales of music like, yeah ......, But I always feel like maybe a bit with like this digital or this kind of like that, it kind of also a bit creates something unique, or a bit like unexpected, or something that you can kind of create, or it's an instrument of its own, but there's not so there's not like this as strict kind of if you play piano or something like a traditional music instrument, because it has such a long kind of history of being that instrument. So you have this repertoire, but here it's more like you are the instrument together with this light and shadow or the sensors. And then maybe it's like an in a way, like more close to how you navigate through world, a bit like, in a way, a bit of being. Maybe I also felt like a whale (laughs), like this echo, you know, like this, like with this sound. And then I. Then, yeah,
did you feel like an instrument?
I was part of it. Yeah, I feel like I was like, I was like, it became kind of one, and I got actually, like, surprised that there was just one time in my head. I was like, I wonder what the time is. I have actually a time, but I didn't want to check it, but like, because I was thinking, can I just enjoy this? So it felt like a flow, in a way, like that. That's really nice.
It's so interesting. how did you feel in your body? Like you said, mentioned dancing. Did you notice that this piece was kind of like making you move in, or affecting your movement in any way?
Yeah, like, I guess, like, in the end, the more I started to get into this, the more I started to think that they are quite far from each other that I would like to reach. I could, like, because I could, I tried to play with my legs, also just if you were like, if I to, kind of like, reach out. But then it didn't work, and then I was like, of course, maybe one thing would have been that I would have moved like this (makes a very wide gestore), so maybe because then I could have played like more like, kind of switched between them more faster, so that I was thinking about the kind of like, but I for sure, I think yeah, with the yeah for in a way that, and then I was thinking like, oh, it would be cool these kind of projects with the dancer to do it, or, like, some, some person who is very in tune with their body, or performance artist, or something that to really be there, like, with this body awareness and the sound awareness and how it's like, moving, Yeah, because it was quite funny. I was just like, having a like, coffee with a friend, and she was talking about, like, she's really into dance hall and dancing, and then my grandmother has been into ballet, and my mother has been into jazz ballet. And then I tried some hip hop at some point. But then I have more been on, like this, raves and stuff, which is nice, but somehow, yeah, there's this kind of easiness, like not a choreography, but more like moving, moving with the music, so that it doesn't actually, it's kind of like improvisation, I think, like, that's very intuitive, and somehow it fits function. Or I was quite excited when I started to read actually what this is, because I was like, oh, okay, this might be quite interesting, because it seems like it's something easily to relate to, kd of So
yeah, okay. It is interesting that you mention raves, I can see this connection too.
But did you think of it as a three dimensional Did you perceive it as some three dimensional object or or like a sculpture of some kind that you could shape? Or did you feel like this act of sculpting? Or did you feel some relation to that? Or wasn't it really triggering anything like that?
I maybe I didn't think of like sculpting like per se, and it's like this kind of I was maybe more thinking of movement and and scale and pitch and but of course, I was also thinking about like this, maybe x, y, z, like this kind of thing. Maybe I wasn't Thinking about this material this kind of thing.
That's okay. I just wanted to, yeah, understand, as it seems like a metaphor anyway, because it's you know, I mean, like, in that sense, it's not a physical material thing. It's more like kind of a thought construction, or, yeah, imagining thing. So it's just like, a description how our imagination works.
But how would you feel if there were other people in the same room with you?
I would be probably more restricted.
Why do you think so?
Or maybe it's like, I will be restricted, or it would take me much longer to kind of, of course, if I would already know them, it would be easier, or if I could realize that they are also, like, kind of on the same kind of wavelength of like, being like, and not maybe I would get to like this, like, a bit like, Oh, if they would try to get too much contact with me instantly, so that we would more like, first explore and then to see if we can play together. That could be quite interesting, but somehow that it would grow over time and not like kind of that now we have to collaborate, in a sense, so it would be more like free. So then, then it would be nice.
I don't know. I think, I think it depends on the person, yeah, okay. How I think it depends on the intuition like, in that I think, like, I don't think it will bother me, like, if they would be very acknowledged about music, I think it's would only be about, like, creating this atmosphere of because when it's about the body, it's also like, we have so many histories with moving with our bodies. So it's also this kind of, like, making oneself feel that you can just play and try. So I feel like this setting of closing the lights or Dimming the lights and leaving the room is like a very good kind of somehow like to give the power to the person exploring that they can just stand still there if they want, or then. But once you add another person here also, then there's this, like a person who is the witness. Of course, I know that you're recording, but at the same time, it feels like a bit more distant than that.
So did you feel like safe in that sense? Yeah,
I think yeah, because, yeah, I think it would, yeah, I feel it was, like, quite funny or enjoyable, and I felt, I felt safe in that sense. So, yeah,
did you find playful?
Yeah, but I think I'm also quite playful in my nature. Or when I feel safe, then I am like, and I can be like, quite, yeah,
okay. Can you describe maybe one moment of this that really stood out for you, like, in a positive way?
I like this kind of boat like movement, like, going slowly up and down. So that was, and I think I was maybe not this organ, but after one a bit more like this, like sound, and then I did this with the two hands, and then I switched one hand up, and like, tried to kind of just barely be sensored. And I really. Enjoyed this kind of like to find out the borders of the where is, how far can I go and and where is this small sounds and how fast, or how fast, but how what is the fastest? You know, like checking the parameters and what is the fastest way of like moving through but yeah, I think I liked also the water sound. Also there was this water and then this, like this sounds, and then mixing them together. So so it got a bit like this. I think I like the most this kind of being in many sounds at the same time, okay? And then I tried to silence it also to kind of stand still for a while and then to continue again. So there was, like this chapters of, yeah, but it's also quite interesting, because, like, if you talk about memory, and then, like, now this, this lamps or sound places have, like, these different characters, but same time, they are already a bit distant, because it's so like, when you are there, the sound is there, but it's just in time and space. And now we are already here. So they are starting to get a bit vague already.
Somehow, yeah,
is there anything you would like to maybe change, or you already said that to maybe that should be closer, yeah,
yeah, yeah, closer and, yeah, I don't know. Maybe it could be like a clear spot, like visually. So you could see the somehow there with the light, you could see the parameters of the like sound somehow, if one wants that in a way. And it could also be interesting. I don't know if some lamps are pointed across. Oh yeah. So you could have like different things, yeah, so they play, instead of playing like up and down, you have maybe playing like this with your like, the movements or something so that,
Oh yes, that would be really interesting. Thank you.
How would you describe your engagement process?
It was quite intuitive in a way that I realized that it's nice to spend time to try to understand what can happen. And so it was like this, maybe explorative, and because, like, I knew what was the project about, but not exactly, for example, the kind of sounds and how sounds change when you do a certain thing. It was this a bit intuitive and also quite playful, I think, like this element of realizing that, and also the fact that it's always something beautiful coming out from what you do, it gave a feeling that it was very, very beautiful to see what, what kind of reaction comes to my own actions. So I would say, like there was this intuitive and playful element and also quite soothing, because it felt that whatever would happen, it sounded pretty great. It sounds it sounds a beautiful sound-world. So it felt also kind of ... liberating, in a way that something pretty would come whatever you .. do.
Before (before the interview started) you used the word "therapeutic". tell me what you mean by that?
Yes, I did use the word therapeutic because I felt it was really this November in Finland, I've been really tired, and it's, it's like, it's easy to get quite much into your own thought and get quite heavy, but I felt quite light in the moment. And it felt that this manipulation of sound and also this discovery of hand gesture and how your movement ... like ... it was a way to relate to your own body in a very, very ... immediate and beautiful way. So it was like a nice way to concentrate on the present moment and it was beautiful to see how, just from my body and my action, something would come out. So there was something I think about presence and about the feeling of also thinking about own body in itself. Yes... and sound, it was also very pleasing. The Sonic experience like at the end, I decided to just stand under the different sound sources. And as the how the sound was traveling around is also very soothing, because you feel surrounded by resonance. So that was also a thing.
which was your favorite sound? Or sound-wise favorite place here?
It's difficult to say, I think, like, in a way ... the most interesting for me was that one, the second one.
The hisses?
because there was this air element, that it felt almost real, which was very beautiful. That's really beautiful. And then it was very pleasant to play this one to the first at the beginning it was, it took me a while, but then it was really colorful, it was the rhythmical element that I found in the end, but maybe the hiss and what was this one here? The water? Yes, I like these two somehow, it gave a nice feeling. But maybe the hiss.
The hiss was also my favorite... But how did you feel or did you feel like you were performing, or .. in which role you thought you are in there?
I thought ... in the beginning, as I said, it was an exploration. I thought it was an improvisation. So there was a performance element, in a sense, that I tried to think about how sounds would be combined together. And so I had a bit this feeling of form, a little bit, from where do I want to go next? And then that was one piece. Then it finished, and it felt okay, that was it. And then I had more time. So then I started to explore without thinking about the form. So it was the two moments. There was a moment of just exploring and moment of being a performer.
But .. so ... did you feel that the fact that you knew that I was recording the sound, affected you to think that you're making a piece?
I thought, of course, because I thought it would be nice to hear. Then I started to think about the piece. But I also took it maybe because in these days I'm playing a lot, because I'm playing in a concert using an iPad, and I'm working with sound in a very different way from how I'm used I mean, this mindset of exploring with sounds and maybe playing so I would have done it anyway, I think to play a piece, because I'm a musician, in a way, maybe distinctive, this thing of thinking, how would I assemble the sounds together.
Do you usually like interactive art?
Yes, very much, very much. And I think I miss, I crave for it. There is not so much.
Do you like art that's kind of like, hard to define what it is?
Yes, I do Yeah.
Did you feel that you were .... in control?
I felt I was, but also not totally, which creates a very good, good feeling that somehow I was in control of something so of like which kind of for how long, or which kind of sound, but there was always this unpredictable element, because we don't know exactly how the sound is going to change and then exactly what the camera takes. So I felt that I was partly in control and partly not, which it creates this surprise element, which I think was soothing. This gives up a bit the expectations and just being, which maybe it was the therapeutical thing also. That you control until a certain point, but I didn't know exactly what was going to happen. Yes.
So if you would change anything, or just be critical of anything .. feel free to say ... how would you improve this?
I would love to do it more and more. I've been just thinking about this is already super great. I'm thinking it's fantastic. And for example, I was thinking this would resonate with so many different kind of audiences,
Also kids, of course, no musicians and just people. It was playful, but it can be serious, and it is so many elements, I just thought maybe it would be nice for in that sense, if one would present it outside to a bigger audience, it would be interesting to have, maybe that you can change the height of the lamps.