{hhr, 24-Sep-2018} There are various references here to Barthes (and Bourriaud mentioning Barthes), and Derrida obviously, as this comes in the wake of post-structuralism. Others extend it to the "corporeal palimpsest", as body-(re)writing. What I find interesting about these kinds of techniques is how you can _invent_/_suggest_ relationships between perhaps very heterogeneous things.

BTW, here's VLC trying to read/write that webm file:

{keywords: [_, writing, rewriting, overwriting]}

{hhr, 24-Sep-2018}

Perhaps also writing as over-writing / palimpsest. Here is a run of a little program that goes through the OCR'ed versions of my text collection to see where 'palimpsest' occurs:

{keywords: [_, writing, rewriting, overwriting]}

{JR, 180921}

Thanks HH, for inviting me into your configuration and into your thinking processes. :-)


Where to start.. well, as I told you over our last voip conversation, I'm at an inflection point in my artistic practice. I was teaching at Leuphana University for nearly three years, helping to set up the practice-based part of a new international bachelors study in digital media. This was an undoubtably enriching experience, where I had the opportunity to meet a number of brilliant colleagues (now friends) and truly engaged students. It was also a physically and mentally exhausting period, commuting between The Hague and Hamburg almost weekly, compiling class plans in airports and at 4am in the AirBnB of the week. Not to mention this being my first full-time university job, which comes with its own strange politics in the German University system. Needless to say, my artistic practice was on the backburner for those years, while my theoretical and conceptual chops grew considerably.


Last year I received a year-long talent development prize/stipend from the Dutch Creative Industries Fund to take the time to establish a studio, and explore new methods, plant research seeds for future work.. and generally interrogate what kind of artist I've become since adding lecturer/pedagogue/media theorist to my list of career subheadings.


The ALMAT residency/configuration comes smack in the early-middle part of this process. So it's a very dear time for me. A time to reflect on my old practice while thinking forward towards the next work.


{keywords: [_, travel, organization]}

{hhr, 03-Sep-2018}

We had already discussed a few topics on the phone. From my reading through your letter, I collected as possibly interesting avenues: Improvisation | Temporality of computation, the time-space structuring of interfaces and algorithms | Sonic-sculptural objects | Failed techno-scientific imaginary | Encryption, compression algorithms | Non-digital systems | Computational ontologies. Furthermore, from your remarks: Ethnocomputing - anthropology of mathematics | Encoding and transmission (quantum mechanics, RSA encryption, perhaps simulations of these algorithms; materialisation of their communication | Language and its ability to represent things

Before going into any of these, I will use the opportunity to throw out a number of thoughts that pertain to all the activities I'm currently involved in.

{function: contextual, keywords: [_, improvisation, encryption, sculpture, compression, materialization, language]}

Initial Conversation

{kind: title}

{JR, 180921}

Thanks for sharing the lovely texts from the Transpositions collection. I especially enjoyed your focus on 'writing', that the algorithm and individual co-write each other. Especially with interpreted and live-coding systems ~ I like to think about literary terminology ~ co-writing, ghost-writing, epistolary writing maybe ~ as there is a context to the conversation which both the interpreter and the artist-programmer keep 'in the back of their minds', so to speak.

Lately I have been thinking, and reading, about oral music traditions in early America, a music scene I was a part of when I lived in North Florida about a decade ago. This is also a tradition of writing, in the sense that the transmission and transmogrification of musical ideas depends on inscription into human memory, and written expression through the recombinative processes of recall. Today most of these songs are collected in gigantic archives like the Smithsonian. There's an interesting tension in what the archive does to the potential future morphologies of the tradition. 

How do the stories we tell in performing become transformed by the writing systems and dynamics of said systems' memory - how do these systems co-construct the process of storytelling? Individually? Collectively? 

{keywords: [_, writing, memory, storytelling]}



{hhr, 05-Sep-2018}

If you have a look at https://almat.iem.at/methodology.html , you can see that our original plan was to somehow "permutate" the configurations in each iteration. We did not strictly adhere to it, in order to react properly to each particular situation, but I think it's still useful as a guideline. In that sense, the roles would be that I work more closely with you in the sense of the practical work on sound, and David would go a bit more into the position of observer and facilitator. (We can still adjust that as we go along). One thing that interests me is what happens when you try to bridge systems; for example, I have been building this computer music environment, Mellite, which I would like to "inject" somehow into the iteration if it allows it. In the beginning of the first iteration with Ron, we were talking quite a bit about it, as there is some common technology involved, SuperCollider, and since you also use, I think, SuperCollider, it may be another opportunity (for me) to use Mellite as a vehicle to understand the processes. The system is quite idiosyncratic, simply because there are not many users except myself, but it has always been thought of as something that could be used by other artists. The process to learn it can be quite involved, as it is based on another programming language, Scala; however, the aim should not be that you need to learn to program this system, unless you are interested to do so, but rather that I can try to work on things that could be connected to your work, even if you use SuperCollider or a different platform in the end. For example, with Ron the situation was that he strongly relies on the pattern system in SuperCollider, something that I had never used, and so for me to understand his composition process, it was interesting to try to implement some of his sketches in my system, and on the way, to formulate my own version of the patterns system. In the end, this became a vehicle for me to understand what happens to a strategy or operational context if it gets translated or transposed into a different environment. What elements can still be "identified", how does the system exhibit resistance, how does the process take specific turns. Another opportunity would be to see, if our working process permits that we simply try to exchange data between two otherwise opaque systems, as audio or OSC signals.

{keywords: [_, pattern, SuperCollider, Mellite, translation, transposition]}

{hhr, 03-Sep-2018}

Although already two years or so in the pipeline, the recently published Transpositions book edited by Michael Schwab contains a chapter by each of us, which could serve as a fast-forwarded into our thought processes on algorithms: https://lup.be/products/108941

{kind: reference, keywords: [transposition, algorithms]}

{hhr, 24-Sep-2018}

aka shapeshifting?

{JR, 180921}

You got me, I'm a SuperCollider junkie. Once Marije Baalman came to Steim in 2011 she converted all of us, and once I became increasingly involved in livecoding that was the nail in the coffin.

One thing I'm interested in developing in Graz is a toolkit for live coding both for performance and composing. I think you, like me, bounce around between discourses and disciplines. So you've probably felt that gut feeling that it's been too long since you've focused on musical craft. It might be a nice exercise to compare/contrast how we create toolkits for ourselves ~ what are our priorities? For me flow, movement and simplicity are important. I would like to experiment more with live storytelling, so it's also a question of what kind of linguistic tools support narrative arcs and accompaniment.


The concept of memory that I mentioned earlier, in the context of storytelling and the oral traditions of old time American music, keeps rearing its head. And I'm tickling the notion of how to integrate different forms of writing/memory media into such a performance. e.g. using reel-to-reel tape machines, hard-drives-as-speakers, machine learning models... with respect to the latter, I'm finally catching up with the machine learning literacy that seems like required knowledge for artist-programmers in the 21st century. ;-) And thinking about user interventions into machine learning systems. I've been taking an online course in Tensorflow to bone up on that. Google's Wavenet speech synthesis & raw audio analysis algorithms look really interesting in terms of their potential to inscribe and speak. There's also a wonderful phenomenon in connectionist systems, called "Catastrophic Forgetting". Besides being one of the great examples of computer science's flair for poetic terminologies, seems also like an interesting entrypoint for user intervention into machine learning processes. 

I'd be interested in exploring ways of making hooks for live coding in SC to connect with some real-time-ish machine learning processes. Could this also be something for us to investigate together? 

{keywords: [_, flow, movement, language, tools, memory, machine learning]}





{hhr, 05-Sep-2018} The "dilation" and "condensation" bits in the figure indicate an idea that we haven't yet tried out well, so it would be a great opportunity to see if it works. The idea is to structure the residency in a way that we begin from a starting point and then let the process up itself up, letting it grow for a while, develop branches or thicket; and then in the second half to counter-balance the tendency of computational processes to "generate" and "multiply", by defining some strategies for shrinking or "radicalisation" (cutting back to some of the roots, removing materials, etc.).

{keywords: [_, methodology]}

{hhr, 05-Sep-2018} One of the main questions in the compositional process that currently go around in my head, is about translation/implementation processes into the abstractions provided by the system, their resistance and limitations, and their interaction. For example, multiple things have now been grafted onto Mellite - it began with a simple representation of sound synthesis processes (similar to a Synth, or NDef), along with a timeline representation useful for electroacoustic composition, radio pieces, etc. I started to bring into it other systems that I had developed, e.g. Wolkenpumpe which is a live improvisation interface, and FScape, which is a non-realtime audio rendering suite which is becoming more and more important for me now as it was redrawn as a UGen based language similar to SuperCollider. Then patterns were added, and I have envisioned them as solving the representation of more higher level organisation, for example as a way to re-implement more ad-hoc programming in many pieces of mine, for example wr_t_ng m_ch_n_ that I presented last year in Riga; this is still high on my list of things to do - see if I can formulate that piece entirely within the abstractions of Mellite, namely patterns. Recently I added an expression language along with the possibility to create simple user-written UI elements; I thought I could use this in a radio piece I'm working on right now, but time did not permit it, so I am again thrown back at writing ad-hoc programs outside my custom IDE. This is for me at the same time frustrating but also challenging, and it's interesting in the sense of observing the difficulty of introducing abstractions powerful enough to be able to embody the kind of sound programs that I need for my works. I think in autumn, I will be working both on patterns and expression graphs, and see how they can be put to work together, possibly with the integration of the DSP of FScape. So when you say, 'language' interests you, this is a point of discussion where we could meet. What does it mean to introduce a language element, how is it informed by our expectations, experience and practice, how does it change the way we go about making a composition, ...

{keywords: [_, abstraction, software, development, framework, workflow]}

meta: true
author: [HHR, JR]
kind: conversation
origin: RC
date: sep-2018

keywords: [brainstorming, methodology, live coding]


{JR, 180921}

Sounds like an interesting method. I've been thinking also, since I'm in this process of digging up and rethinking my own artistic practice (and methods) - perhaps we can both reflect on how we, individually, create work? An old painting teacher of mine once told me "people come to see artworks to see decisions". I took this as a message on parameters, what are the parameters we set for ourselves in a specific work, or even more generally, as a creative person? I would be interested in exploring how your concepts of configurations and mattering in algorithms can also be used when developing individual methods. A thicket of methodologies, parameter sets, and some etudes along the way.

{keywords: [_, methodology]}



{hhr, 05-Sep-2018}

Some abstractions in Mellite:

  • Proc: a real-time UGen graph, very similar to SuperCollider's SynthDef / NDef
  • FScape: a non-realtime UGen graph, allowing us to do things which take much longer than real-time, or perhaps run much faster. Random access to sound or image streams. Open questions: How to represent structures that are not 1-dimensional signals? How to add "control structure" elements, such as conditional execution, branching, repetition, "mapping" over collections, resetting and restarting sub-graphs. How to properly represent "lazily" created sounds, to cache them, render them "on demand", ...
  • Pattern: a stream-processing graph, pull-based, with similarity to (Scala) collections. Open questions: how to couple them to push-based structures; how to introduce asynchronous sub-graphs that require FScape rendering or other things to happen that take time.
  • Ex (expression) / Widget: a pull-based graph for "values", similar to functional reactive programming. Plus "actions", imperative and interactive things that bridge expressions with UI widgets. How to couple that with patterns, and how to enable it to run Proc runs and FScape renderings.
  • OscNode: an abstraction created rather ad-hoc for the Koerper project. Falls down to "plain Scala". Questions: could/should we use an embedded language here as well to avoid exposing the entire surface of Scala (with problems of future stability and compatibility), how to couple that with expressions/widgets, or the translation into real-time data streams for SuperCollider UGens

{function: description, keywords: [_, Mellite, abstraction]}.


{hhr, 24-Sep-2018}

"Flow and movement" are two terms that also resonate with me, perhaps in one way because I'm still interested in understanding Deleuze/Guattari's concept of machines, and in other way since movement seems a good replacement for otherwise cybernetic terms which have too much historic baggage and tend to direct the discourse in unproductive directions (e.g. talking in terms of "movement" instead of "control" seems more fruitful to me). Flow and movement also inherently stress a process-over-product perspective.

Funnily, simplicity does not immediately capture me, although David has worked with the term 'radicalisation' which may relate to it. Somehow I tend to make things that eventually are complicated, messy, buggy and slow, and I somehow then "accommodate" those properties, even if they seem to make life unnecessarily burdensome. So simplicity could be an interesting "frictional term" in our discussion.

Likewise, storytelling and narrativity are two terms that cause some "repelling force" in me, so it could be interesting to discuss our understanding of these as well. Perhaps I should lay out where that preoccupation comes from. I think it comes from the "spatial" property that for me characterises installation art. In my adolesence I was very much fascinated by the collection of the only contemporary art museum within proximity which was the New Museum Weserburg in Bremen, which had (has?) a focus on 1970s and 80s, at least as I remember. That building for me was one giant breathing space, so calm and not "telling" me how to move, where to go, how long to stay, what to read. From the gray blurry photo paintings of Richter to the objects of Beuys, a metal swing by Horn, an inconspicuous sound installation by Julius; and next door GAK, the society for current art, with more recent works by less famous people. Somehow, sound installation for me presented itself as "against music" in the sense of "against stage", "against dramaturgy", "against narrativity and linearity". For me the term narrativity is always connotated with consumption, e.g. the successful sale of a movie, the "success story" of a company/product, the elevator pitch, the TED talk, etc. that is the "formatting" of something into things easily digestable and "communicable" to an anonymous mass audience. Also, the fallacy of distinguishing form and content ("authoring a VR scene", "scripting an experience", ...)

That does not mean that narrativity, from my perspective, cannot be a useful concept, but then the question becomes how we define narrativity, how is it constructed (and by whom). I am very much for "broken narratives", in a way that can only be understood in the plural, and that originates from the person that experiences a piece of art (or any aesthetic encounter).

Notwithstanding, I am very much interested in text and language processing. Of course, NLP is an exciting field to explore, although I am kind of wary with respect to the imperative "required knowledge for artist-programmers". Since a while, I am sensing a strange sort of force exerted on artists working in elecronic media that urges them to navigate in this new landscape of open source neural networks, in the sense that it seems to be required to be have some python, tensorflow, you-name-it, on their CV, no matter what they are actually doing with these systems, just to prove a certain "literacy" in these technologies. I will be honest: I have tried some of these things, and I have a fairly good understanding how these work in principle, but I would never describe myself as any sort of expert on neural networks or machine learning systems. I'm deeply sceptical when someone comes and presents themselves as working at a "forefront", because in 2018 the "best" technology is to encode things with Generative Adverserial Networks, possibly attached to some body fluid and a blockchain cryptocurrency, just to throw in a few buzzwords. I'm deliberately sarcastic here, but the point I want to make is to say that I think as artists we should question the pressure of technological "novelty" to subscribe to; we shouldn't be afraid to pick any sort of machinic approaches, whether they were invented in the past, or in the past six months. We had a funny little off-discussion with Frieder Nake at the last xCoAx, who was very critical (I think rightfully so) after a keynote that basically consisted of a romantisation of "deep" convolutional image processing. I am not against these processes, but I think we should ask ourselves first what it is that makes them interesting, which properties, and then how to appropriate them as artists, with no pressure to "correctly implement" them or to use the fancy toolboxes instead of coding up our own clunky partial solutions.

To come back to NLP, I think it is very interesting to probe the collision between sound and text/language, the former resisting the encoding in speech and language attributes. What happens when the irreducibility of sound comes together with the transcoding into phonemes and formants, intonation, words, sentence structures, then cross-linking between the linguistic properties, etc. If that can result in "broken narratives", I am very much interested in that. I have only rudimentary experience with NLP, I mostly use old fashioned signal processing to work on the acoustical level of spoken language, and I use simple (ha!) processes such as look up, permutation, chunking and re-sequencing, to work with text, and I definitely would like to learn more. For me the challenge here is to break up the (supposed?) discrete nature of language. I have often said, it would be fantastic, if we could write a reverberator for text. Like you enter a text fragment, and the machine "reverberates" that, creating echoes and decays, blurring, ... These utopic kind of machines I find fascinating.

Perhaps David wants to add another perspective on these questions? I know he is very interested in the design of a new programming language.

{keywords: [_, movement, flow, simplicity, storytelling, language, code, writing, text, technology, computer, machine]}

{hhr, 26-Sep-2018}

Not sure if this is of relevance, but since I was just (in other context) reading through Massumi's "Autonomy of Affect": "two languages: two dimensions of every expression, one superlinear, the other linear. Every event takes place on both levels—and between both levels, as they resonate together to form a larger system composed of two interacting subsystems following entirely different rules of formation." (p. 26; superlinearity = suspense (suspension?)) ; intensity as (inter-/dis-)ruption of narrativity; the other associated with _expectation_, which is guess is what I wanted to say when describing my experience of walking through the Weserburg: absence of expectation.