{date: 201125}

I have talked and written about FScape multiple times now. It is the oldest (ca. 2001–) software I created that I consider a “serious” experimental sound software. I also keep coming back to it. There was something of a simplicity in it that allowed many people to quickly get started, and so it is also the software I receive the most feedback from, as people write to me from time to time, speak a compliment or ask a question.

The source code is “brute” in a way, a plain imperative Java code that reads sound files, mangles data, writes sound files. There is no greater “form” in the sense of an underlying signal processing framework involved, it just is really plain operations. That is its beauty, but also makes it problematic, as I have moved to other questions and code style, and while I maintain the “classic” FScape program, I rarely add any new signal processing routines.

Instead I started to think that it would move the project forward, if I found an abstraction and framework to integrate it with SoundProcesses. The repository is called “FScape-next”, and it became something both similar and dissimilar to the classic program. It uses the UGen approach already employed by the real-time synthesis system within SP. Was this a good choice? A good choice would be one that both makes it possible to express most of the modules in FScape classic, while also bringing in a fundamental research layer that makes it a critical piece of software.

{group: fsc1, keywords: [_, experimental, simplicity, operation, beauty, style, UGen]}

The UGens are implemented on an asynchronous and hybrid push/pull streaming library (Akka Stream), and the behaviour is actually quite different from the synchronous API known from a real-time system such as SuperCollider server. Packets could arrive in different orders and frequencies, there is a nominal block size, but in principle block sizes and data rates can vary. For instance, take the UGen ResizeWindow(in, size, start, stop) that allows to resize packets (windows) of signals, essentially increasing or decreasing the data rate compared to other signals, or the UGen FilterSeq(in, gate) that retains only those elements in the stream for which the corresponding gate signal is positive. On the other hand, it feels like I'm still subjected to the linearity of the passage of time that is common to real-time processing. There is something missing, there is no good generally available random access mechanism (even though some UGens perform internal buffering and run random-acess operations).

Somehow I am hoping that this loss is compensated when UGens are more clearly typed (currently they support three different numeric types which are transparent to the user), and when a form of reset / reuse mechanism is introduced, so that more "procedural" code can be written.

On the right, that's how I sometimes sketch a small algorithm that I want to employ for a sound. This is from an ongoing electroacoustic composition Shouldhalde, and I have a section of one of its parts that I'm not happy with—the sound is still a quite raw microphone recording, you hear the acoustics of the studio, it produces the “wrong space” so to speak. I bounced the 8 minute section, and the idea is to reorder all the sound particles in it, in a variant of what in classic FScape is the StepBack module. That module does a segmentation of a sound, based on local minima in autocorrelation, and then reorders the segments in reverse order. What was developed in the sketch, can be easily formulated in language: Go through the sound. There is a minimum segment length, below which no cutting is down. After the minimum length has been reached, start cross-correlating the sound with “the remaining chunk” of the sound, stopping after a given maximum duration. Determine the local maximum of the cross-correlation; cut the sound here, and proceed by jumping to the correlation partner. “Remove” the parts of the sound which have been “used” so far, and repeat. That's a simple description, but difficult to formulate with the given UGen infrastructure. It requires structured data; for example, one has to maintain a list of regions of the sound that until now have not yet been used. After each step, the used region must be “removed” from that list. And we need random access to the sound. I could implement the whole algorithm as one new UGen. But that goes against the idea UGens as smaller reusable atoms instead of more complex compound algorithms.

{group: fsc1, keywords: [_, UGen, asynchronous, behaviour, signal flow, time, acoustics, electroacoustic, segmentation, auto-correlation, chunk, formulation], artwork: Shouldhalde}

There are three stategies then. First, consider the effort too high and uneconomic—the time away from the primary artistic process is too high, the risk is too high to lose the momentum on the artistic work; thus choose a different algorithm, or abandon the action, and turn to another action. Second, implement the algorithm in “plain code”, ignoring FScape and its current restrictions. That will cost a medium amount of time, the downside being that it will become a particular solution that can not be integrated with the existing code. Third, interrupt the work, and try to conceptualise and implement the necessary architectural changes to FScape. A bottomless pit.

{group: fsc2, keywords: [_, strategy, absence, abandon, implementation, interruption]}

With software written by yourself, there is ultimately only one instance to blame for its deficiencies. It never works to the full extent, so you have to live with this precarity.

{keywords: [imperfection]}

A fourth strategy is to work around with other abstractions already in SP. I could run one search iteration go in an FScape program, and then write an enclosing Control program that does the buffer erosion and re-runs FScape again and again. This is similar to the algorithmic loop employed in Writing (simultan).

{group: fsc2, artwork: WritingSimultan, keywords: [_, strategy, abstraction, buffer, loop]}

{date: 201208}

So now I have a working algorithm for performing the “step around”. I always noted the feature of audio files as the input and output of processes being their generic connectability or operational closure to use the Varela/Luhmann'ish term. The audio file does not reveal any semantic or structural information by itself; the algorithm that processes an audio file in many cases does not care whether that file contains a fragment of speech, a field recording, a synthesised sound, etc.

But there is also a downside to the “flattening operation” of bouncing. At this stage, I am not decided whether Shouldhalde will be a “stereo album”, or multi-channel electroacoustic concert pieces, or “material” for a variety of forms (something I am trying to pursue as a project on its own, moulding it into hybrid forms between electroacoustic music, improvisation, and installation). In any case, the spaciousness of the timeline structure of Shouldhalde would disappear if I mixed all signals together to a stereo file for processing in “step around”.

{persons: [Niklas Luhmann, Francisco Varela], artwork: Shouldhalde, keywords: [_, file, connection, hybrid, timeline], group: fsc5}

{date: 201208}

Therefore, I created different buses and distributed the materials so that there are no temporal overlays within a single bus. A bit like placing things on different tracks, although I rejected the notion of fixed tracks in Mellite. Still, I used colour marking to better see which are the bus assignments. For this example, I needed four stereo buses, resulting in an eight channel bounce that could be reorganised by “step around” (the algorithm uses a mono mix to calculate the correlations).

{group: fsc5, keywords: [_, colors, correlation]}

{date: 201208}

Now to put back individual regions, I want to segment the output from “step around” based on silences. This is more or less easy to describe as an informal algorithm:

    Algorithm:
    - to section a file into regions of
      signal versus silence
    - there are low and high levels,
      based on magnitude of sample values
    - there are minimum durations of low
      and high periods; i.e. in the output
      there shall not be any silent period
      shorter than a given duration, and
      no active region shorter than another
      given duration. This is achieved by
      "absorbing" low regions, e.g. if the
      minimum duration for low sections is
      100 ms, and in the analysis there is
      a low section of 50 ms, that section
      is absorbed into the surrounding high
      section.
    - high sections trump low sections, i.e.
      we do not absorb high regions within
      low regions (we do not "omit" signals).
      If a high section is shorter than the
      minimum high duration, it is extended
      to the minimum high duration; the onset
      stays the same, but the region's end is
      moved forward in time. For example if
      a high region starts at 4.0s and lasts
      for 0.1s, and minimum high duration is
      0.5s, than the region begins at 4.0s
      and ends at 4.5s (instead of 4.1s).
 
Making that operable in imperative code would be simple. But in terms of signal flow in FScape, that is quite difficult without knowing ways and tricks to formalise these steps and constraints.

{group: fsc5, kind: pseudocode, keywords: [silence, segmentation, duration, constraints]}

{date: 201210}

Like so often, a simple UGen was missing that would make this anything but a convoluted translation. I sketched out on paper that a block would be useful that keeps a gate or trigger signal “high” for a period of time. I call this new UGen TrigHold, even though I have moved away from “trigger” signals altogether in favour of “gate” signals; they can be used to trigger something at sample level precision, whereas a trigger can only occur every two sample frames.

{keywords: [_, UGen, translation, trigger]}