JR
I was really curious about the wavenet algorithm that google released because it looks like it can be hackable. This is usually what interests me: how to get inside and, really, kind of use and abuse it. Of course it’s a large and ambitious thing to start working with a platform like this, but I’m already curious about just doing some small experiments. I was building a catalogue of techniques and experiments for digging into this algorithm and the model that it builds.
HHR
I must say I was really frustrated because, unless you have real processing power, the amount of time into training that thing is like incredible. I mean, with my normal laptop I gave up trying to converge to something that could remotely reproduce some of the corpus. Maybe I did something wrong, but I was really frustrated about it. I think in many examples you find in their website the system is trained on huge databases, with huge processing power, for a long time. And in general, this is the kind of frustration I have with a lot of the neural network stuff. Of course, if you download the ready made models they are beautiful. They work very nicely. But if you start to build up from your own corpus and try to train those things, that takes a lot of time to process. Maybe if you have all the GPU libraries, I don’t know. I never got that to work, so for me it was all just very very slow.
HHR
I'm kind of interested also in WordNet, but the database is only for the Englsih language. So I was looking for a german version of WordNet. I found out a similar thing exists, it's called GermaNet. It basically wotks the same as WordNet, but the idea is to use the German dictionary. I touched it only superficially but I thought it was really nice. For example you can calculate the distances of relationships between two words, if you move up down the morphology. Or you can discover how many synonyms you need to jump to get from word to word. So I’m kind of interested also in this "word processing thing" that has nothing to do in the first place with sound. But maybe they can be combined as well, I don’t know.
JR
Yeah. Well I’m thinking about how these training processes and representational models could become content-based, from a compositional point of view. How they might become the form and content of the piece, rather than what you said in your comments about "AI" being the latest buzz-word in a long list of tech innovation lingo. I'm more interested in looking at the aesthetics of the infrastructure, the process, and through that get to some kind of cultural revelation.
That’s part of what leads me to draw machine learning together with the American oral tradition of music. I feel there are connections between the way the oral transmission of music worked in this case and the way these machine learning models "morph" over time. And I’m thinking about that: the training process itself being a sort of meta-artefact. What about creating a piece that evolves over multiple performances, where in each performance there’s an aspect of gathering information through collaboration and transmission, that adds to the corpus. And then the model trains over night, or in a week, or whatever it takes and then the next performance has kind of like transmogrified or like mutated or shifted. Musical form or musical patterns, which then live musicians perform with, or the audience even is involved in a participatory manner.
And then you have this body of information or this model, that is the piece, that’s just almost orally transmitted. It follows this pathway of going forward, you know? Maybe the model gets posted online and people can branch it into different performance streams. In a similar fashion that a melody can go from being an old pirate song to becoming a gospel motive. Then it gets its own lineage in the gospel tradition, and then maybe it gets caught in a dance song, gets its own lyrics and then takes off in that direction. I really like oral music and story traditions as a metaphor to think about machine learning systems; as bodies of memory that undergo transmission and transformation.
HHR
It’s funny: you could even generalize that, beyond machine learning stuff. If I think about one particular rendering of this idea, it reminds me of what I was thinking about when I was doing my thesis, a couple of years ago. It was also about the idea of how the creation of a piece can kind of be reflected in the piece itself. In a similar way, understanding pieces as databases, as data containers that can change over time. And then often, when I do installations, I’m interested in incorporating some events that happen during the exhibition. Data that can be gathered and integrated, conttributing to a continous re-writing of the piece, in some way. I think that’s a similar idea, or kind of the same idea. In your case you are talking about training models of neural networks or some similar structures, but the general idea, in a way, is the question of how a piece might be conceived as something that evolves over time. Something that doesn’t reach a specific halting state, that then you perform and it always remains the same. You can question the boundaries of a piece, in a way.
JR
Yeah, I think the key thing there is how the piece can have memory. How there can be a history and a memory that travels with the piece? And it doesn’t have to be a machine learning model, it could be, like you said, a database. It could be reel to reel tape. You know, you were talking about palimpsest. Like a purely recorded reel to reel tape, where you still would have some of the leftovers of previous recordings. I mean, I also have this kind of fetish with "media archeological" thinking about different old media and new media. I think that part of what attracts me to neural networks is that it is one of the most contemporary imaginaries of what memory is. So, for me, it is connected to this whole network, or rhizome, of memory technologies. You know, like databases, hard disk recording, cd recording, reel to reel recording. I posted an image in the research catalogue of a william’s tube. It’s the first digital memory device. It’s like a CRT tube that maintains a grid of dots. It’s literally like a CRT and it maintains a grid of dots that it’s continously reactivating the phosphor on the inside of the tube in order to use the properties of the phosphor to hold the charge a little bit, to hold the memory. And it’s constantly rescanning and reactivating the phosphor. Definitely the first digital memory was made out of banks of this phosphor tube.
HHR
I remember something similar with sound, like a delay line memory was also used in some synth that is probably more prone to interrupt, you know, noise from the outside. You need some medium in which something needs time to travel basically, to keep the memory alive. That’s an interesting concept.
DP
Talking about memory, there is a colleague of us, Thomas, who told us a story about how the RAM was implemented in the very first computer. They had some sort of bits, metal bits that you could magnetize in two directions, storing one bit basically. Then, in order to read the bit, you would need to know what kind of magnetization it had. But you don’t know that a priori. So the solution is to write it once again, and see if it changes. If it doesn’t change, then you know that what you have written corresponds to the value that previously stored. If it does change it was the opposite, and you need to write it once again, in order to maintain the memory. I found this very interesting. The idea of memory as a process of re-writing. You have to perform an action in order to know the stored values.
JR
I can imagine. I was reading your entry in this artistic research book.. this is a bit like ‘you can’t perceive the system without affecting it’.
HHR
And you also need to interpret the memory. I mean, memory doesn’t exist outside your interpretation. Somebody else might interpret the same memory completely different. So the context changes, that’s why historical science is always continuously re-writing the interpretation, because you would say that the data is the same, but it is not. It’s always newly interpreted in terms of what you know now, or what you think the context is now. So in any case, memory is never stable.
JR
Yeah, it’s interesting to think that we actually maybe don’t even have the technology or the interface to really read the memory anymore. If you talk about history, we don’t have the context. The context hence could be a metaphor for the technology or the technology could be a metaphor for the context. We’re dealing with an obsolete context.
HHR
Recently we had some people from early music giving a presentation. That’s an interesting issue, this performance practice of old music. What's the meaning of performing ancient music? You have the original instrument but you don’t have the original society, so what happens to the music then?
JR
Exactly. I mean, it’s the same with early american folk. You know, a lot of the music that is part of this canon, especially the solo singer variety, let's take Roscoe Holcomb as an example - has a very ‘lonely’ sound to it. That loneliness is in some cases capturing a kind of embodiment of the geography where the music lived. Wide and sparsely populated geographies; in the case of Holcolmb and the traditions of the Appalachian mountains that you could now maybe call "proto-bluegrass", you've got singing styles that are even closely entangled with traditions of hollaring that were only possible in places where you would need to be able to send out a sharp and far-travelling shout into the landscape.
This music has had a few booms since recording technology became available widely enough. In the 60s you had the big folk revival, which in a way echoed a few decades of earnest and idealistic musicological work documenting the folk musics of the United States. In the early 2000s there was another mini-revival, there’s maybe even one going on right now. Each 'revival' was an attempt to reinterpret and retransmit these musical traditions, their character, unique and idosyncratic playing techniques. There’s the attempt at de/re-imagining the social context as well. I don’t know if it’s the same in old European music, were it often seems that there's an attempt at recreating the context rather than transmutating it.
HHR
I think that’s two diverging possibilities. There’s one tradition that tries to say ‘look, we cannot recreate the original situation, we try to make sense of it’. And the other one that tries to be authentic, in a way.
JR
Yeah. Especially the desire to recreate the context, there’s something really weird about that.
HHR
Well, you have to embrace the fact that something hybrid will come out of it, and then that’s ok. If you integrate that into your whole endeavor, then probably it’s ok. If you pretend that you are going to recreate some original experience then probably there’s something wrong in the conceptual outline of this project.
DP
With regard to your text, which I didn’t have time to reply, I have some ideas. They are quite unstructured.
First of all, I have a little bit of a problem with neural networks and artificial intelligence algorithms. The problem is that the narrative they bring with them is so much stronger, compared to what they actually can do, or to what they do. There is a big disconnection for me. When you say neural network everyone, including us, has a specific ‘feeling’, or a specific image of what the thing does, of what the thing should do. This expectation is never fulfilled by the algorithm itself. Morevover many algorithms use actually very simple digital signal processing, recurrent processes like convolution etc. In my opinion this simplicity, and the algorithmic blocks that compose it, get strongly overshadowed when you say ‘this is a neural network’. The neural network part is actually a strong interpretation of what the algorithm does, and how it works. This is one part of the problem.
The other part of the problem is that usually neural networks are used to solve problems. They are trained to converge to a point. And this is for me a little less interesting. I like things that do not converge, because this shows a bit more about themselves, about how they behave and how they cope with difficulties. And I think you might share this idea: also with respect to your idea of the piece about a neural network that does not converge, but continues to evolve.
JR
Well, definitely. For me it’s really important to challenge this notion of fixity. Which I guess has to do with this idea of convergence, that at some point you have a solution. I would like to try to see how this process of training could be kept in a processual state. Which is why this idea of catastrophic forgetting I found so inspiring. Because it’s seen as a huge problem, it keeps you away from converging to a solution. But actually it’s a great entry point. I was thinking about how you can keep the model in a state of becoming, rather than letting it turn into something fix. So I totally share this feeling. To me having a trained model that’s just classifying things in a very predictable way, where is the critical engagement? Where does it become artistically interesting? Where is it providing novelty? Because, I mean, this kind of classifiers most of the time I feel that if I just intervene as an artist making decisions, that’s more interesting than if the network itself was making decisions.
DP
Using the words of Agostino Di Scipio, there’s a difference between interacting with something and being used by that same thing. When you are not even critically engaging with the algorithm itself, it means that you are accepting this narrative and using it as a tool and, in that moment, you are as well used by that tool as you are using. This, I think, is an interesting way to think about also languages.
CR_01h28m
2500w
1s / 30 w
{persons: [HHR, DP, JR]}
⌕ thinking of machine learning in terms of a body of memory that gets transmitted
⌕ memory and interpretation
⌕ memory and context
⌕ memory and action