(C) Web-mediated duo improvisation———————————————————————

26.03.2020 - 16.04.2020

Shifting to web-mediated audio involved the preliminary issues of signal transfer and latency. We trialed two audio streaming solutions, Cockos Ninjam and Audiomovers Listento vst-clients, and settled for the latter for the shorter latency and larger choice of audio compression formats.

Experiments in web-mediated musicianship developed the ‘cadavre exquis’ strategy that consists of a simple seed audio material being streamed between the players, and both simultaneously processing it, as shown in videos 17 to 18. Then, we switched to the ‘four-hands modular’ strategy previously explored (see videos 1 to 11), but this time with remote control of the modular synthesizer routed via User Datagram Protocol (UDP). An added feature in regard to the previous web-mediation was the addition of supplementary video feeds. The whole video feed was multiplexed using Zoom.us, as shown in video 19.

The final sessions explored the idea of a graphic monitoring of the sound, which proved to be the major finding of our web-mediated experiments. The Izotop Insight plugin includes a spectral histogram of user-definable length, allowing to perceive the shared audio stream as a constantly refreshed topographical map. Video 20 shows the visual feedback window for an improvisation on a single shared signal path. Video 21 displays the same system with two signal paths, adding more sonic complexity.

(B) Four-hands electroacoustic deejaying———————————————————————

05.03.2020 - 12.03.2020

The following exploration included digital audio processing on pre-recorded material, investigating the potential of a digital performance tool specifically designed for dj work: Pioneer DJ System XDJ-RX2. We were able to conduct four experiments with the Pioneer dj controller before the Covid-19 pandemic enforced a shift to social distancing and web-mediated duo work.

The work started from duo deejaying on pre-existing tracks in order to get accustomed to the interface (Video 12). We then moved on to duo-deejaying using our own sonic material, but found that the approach resulted in limited interconnection (Video 13). The idea of the cadavre exquis form emerged as a response: could more connection be found by sampling and manipulating each-others' sounds? (Video 14). Finally, a ‘dynamic cross-sampling’ system was patched and programmed, allowing for each player to record the other’s output at any moment, and use it immediately as a basis for his own sound work (Video 15).

(12) Four-hands DJing I

———————————————————————

05.03.2020

Idea

Stage zero: Playing/testing the controller with the default sounds.

Hardware + software

Pioneer DJ System XDJ-RX2 -> Rekordbox software -> Stereo out

Observations

General:

Very good impression of the controller but no connection with the default sounds.

(16) Web-mediated / ninjam duo

———————————————————————

26.03.2020

Idea

Exploring an online situation to make music together. How can we continue with the work during covid-19 lockdown? Ninjam Test.

Hardward + software 

Otso: Reaper -> Synthesizer (Kontrol-Native Instruments) -> Ninjam send/receive plug-in -> Alejo.

Alejo: Pioneer DJ controller XDJ-RX2 (remixing Otso’s music) -> Reaper -> Ninjam send/receive plug-in -> Otso.

Observations

General:

Surprisingly good feeling. Acousmatic listening. 

Long latency, good for sonic improvisation where no synchronisation of events is needed. Not very useful for real time interaction between musicians. 

Each musician hears a different mix (different track shifting or time shifting regarding each other, individual track gains).

(13) Four-hands DJing II

———————————————————————

05.03.2020

Idea

First step: Playing the controller with each player using their own sounds as source material.

Hardware + software

Pioneer DJ System XDJ-RX2 -> Rekordbox software -> Stereo out

Observations

General:

Overall, it felt like playing side by side, but limited interconnection. The sound and intentions did not really mix. 

Summed into a stereo mix - sometimes hard to distinguish each player's sounds - confusing since we were controlling our own voices - presupposition to hear it (in order to control it).

Idea: “Cadavre exquis” - Dynamic sampling of the other  - Next step will be that Alejo sample Otso’s output and use that as material to play. This could connect us more.

Otso: Intimate connection to the sound (being my own sounds that I know well), instrumental feeling – Own voice as if playing the guitar, since the sounds were so intimately familiar. 

Alejo: the controllers feels very good and is intuitive. Definitely made to play live music. I especially liked how it feels the deck/scratch surface and the big tempo fader.

(17a-17b) Web-mediated / cadavre exquis

————————————————————————————————————————————————

01.04.2020

Videos 17a and 17b are from the same moment but recorded at both ends of the transmission channnel, demonstrating the different monitoring situation between performers.

Idea

Web-mediated cadavre exquis. An initial seed sound is played by one musician, sampled and processed by the other musician. The process can occuer it in series or in parallel (one after each other or at the same time processing the same source). 

Hardward + software 

Otso:  Korg midi controller -> Ableton Live (Click sample) -> Listento send/receive plug-in   

Alejo: Reaper -> Listento receive plug-in  -> Otso signal -> Pioneer DJ controller XDJ-RX2 input -> XDJ-RX2 EQ -> XDJ-RX2 Sound color FX -> Listento send plug-in.

Otso and Alejo: Microsoft Teams -> web video call (audio and video).

Observations

Otso:

Verbal communication issue. 

Communication channels: audio (software audio) / speech / visual.

Communication between spaces (Headphone space - Otso vs Monitor room - Alejo) becomes pipelined into channels.

Making music in physical presence is a 'complete' situation (Gestalt): internet compartments it.

Camera positioning becomes important.

Needs practice! 

There were promises of moments of genuine musical connection. 

Shy and overwhelmed by the technical setup (including the recording, monitoring, actual music making) - the tech situation is heavy, but nevertheless - there is a seed of shared musicianship. 

Alejo: 

Second online session: system test.

High signal to noise ratio: Teams audio was mixed with Listento receive.

Each acoustic space brings its intrinsic sonic information. 

Unpredictability of the sound material has a positive feeling.

Slow beginning, need to find first something to play with the incoming sound.  

It helps to start with an easily identifiable (= simple) seed material.

Good performance feeling with the control switch of XDJ-RX2 Sound color FX.

(3) Four-hands modular / additive synthesis III

———————————————————————

05.11.2019

Idea

Two-person control of modulations on a single signal path with one capacitive sensor-based control interface (Make Noise Pressure Points).

Hardware

Modular synthesizer

Sound Source: 

1st oscillator (sine, saw and triangle) -> filter -> LPG -> VCA.

Modulations: 

2nd oscillator (sine) -> 1st oscillator FM (1v/oct). 

Pressure Points (press out 1 and 4) -> 2nd oscillator FM (1 v/oct) and filter (FM). 

Pressure Points (press out 2 and 3) -> filter (quality and span).

Pressure Points (T-gate) -> ADSR (env in) -> LPG (cv in).

Observations

General: 

Territorialaised experiment - Alejo plays with 5 knobs and 2 switches - Otso plays with 4 pads of Pressure Points.

Linear signal.

Alejo: feels very good to play with the resonance of the LPG. 

Otso: pressure points feels better than Akai and is more precise.

(14) Four-hands DJing III / cadavre exquis 

———————————————————————

05.03.2020

Idea

Next step: Playing the controller with Otso having his own sounds and Alejo sampling Otso.

Hardware + software

Pioneer DJ System XDJ-RX2 -> Rekordbox software -> Split each player signal into double stereo out -> Quadraphonic out (back/forth). 

Observations

General:

Double stereo makes clear what we are doing and we gain spatialisation. 

Use of “Master Rec” in Pioneer DJ System XDJ-RX2 to sample Otso - Smart looper joke!!! Smartest looper around!

Learning the controller better makes us more versatile and precise. 

Arriving to the limit of the machine as it is designed. 

Otso: coherency in the sound, coherent development from one state to the other 

Alejo: Possible to be very similar to the other, once I had several recordings I could choose from earlier recordings – move back in time, create contrast, choose to be close (in time) or further). Slice the other’s performance. It creates a relation between the players and the materials.

(4) Four-hands modular / additive synthesis IV

———————————————————————

05.11.2019

Idea

Two-person control of modulations on a single signal path with two control interfaces (Akai pad and Make Noise Pressure Points).

Hardware

Modular synthesizer

Sound Source: 

1st oscillator (sine, saw and triangle) -> filter -> LPG -> VCA.

Modulations: 

2nd oscillator (sine) -> 1st oscillator FM (1v/oct). 

Pressure Points (press out 1 and 4) -> 2nd oscillator FM (1 v/oct) and filter (FM). 

Pressure Points (press out 2 and 3) -> filter (quality and span).

Pressure Points (T-gate) -> ADSR (env in) -> LPG (cv in).

Akai pad

Modulation:

1 pads -> Midi to cv -> VCA (cv in)

Observations

General: 

Territorialaized experiment - Alejo plays with 1 pad of Akai - Otso plays with 4 pads of Pressure Points.

Linear signal.

Alejo: Akai’s pad feels good to control the VCA but is not very precise and can't stay at one specific value. 

Otso: different physicality, more engaging, gate control is a power position (determinant on the sound output).

(15) Four-hands DJing IV / dynamic cross sampling

———————————————————————

11.-12.03.2020

Idea

Moving further: Both performer starting playing with their own sounds, after both players are free to sample each other.

Hardware + software

Pioneer DJ System XDJ-RX2 -> Rekordbox software -> Split each player signal into double stereo out -> Quadraphonic out (back/forth).

Dynamic cross sampling Max patch:

Pioneer DJ System XDJ-RX2 -> Rekordbox software -> Split each player signal into double stereo out -> Send to Max software -> Max patch to record both players output (two keyboard keys to Rec/stop each player) -> Recorded material loaded back to the Pioneer DJ System XDJ-RX2.

Observation

Methodological victory!

Only by going through the previous step we could actually imagine a cross sampling system on a DJ controller! 

Research – Creation in essence. 

Technological feasibility (1 day of work) - going beyond the pre-designed use and features of the machine. -> Tech hack and creative use of tech combinations. 

In this case, serial use of software on one sound card and creative signal routing. 

Cross sampling Principle: each can play & record the other and use the material. This enables a sonic discussion, and retains the qualities found in the previous step: coherency in the sound, coherent development from one state to the other, move back in time, create contrast, choose to be close (in time) or further.

Simple idea, Simple metaphorical image (cross arrows). 2nd important principle that we find!! 

Example of electronic instrument plasticity, creative signal routing/mapping rather than create a novel instrument. 

System needs practice! - Absorbed in the interface and its (novel) functions, it’s hard to keep track of the other. 

Listening!!! It becomes crucial to listen to the other -Intersubjectivity potential - but hard to attain - needs practice. 

Via listening – music starts to happen. 

Potential for virtuosity in this practice (or un-virtuosity).

(18a-18b) Web-mediated / dynamic cross sampling

———————————————————————————————————————————————

02.04.2020

Videos 18a and 18b are from the same moment but recorded at both ends of the transmission channel, demonstrating the different monitoring situation between performers.

Idea

Seed signal/first trigger (Otso) -> web streaming -> processing (Alejo) -> web streaming back -> processing (Otso)-> web streaming back -> etc..

Hardward + software

Otso: Korg midi controller -> Live (seed signal: drum machine click with random rhythm) -> Listento send/receive plug-in -> Alejo signal -> Live (sampling Alejo) -> etc..

Alejo: Reaper -> Listento receive plug-in -> Otso signal -> Three Sisters filter (All input)-> Erbe-verb reverb (input) - > Listento send plug-in -> etc..

Alejo - modulation:

Pressure Points (press out all) -> filter (v/8 Freq).

Pressure Points (set cv out) -> filter (span).

Otso and Alejo: Microsoft Teams -> web video call (audio and video).

Observations

General:

How to document the situation? – the control gestures, parameters, interface 

Mystery of the other’s setup, processes and gear. Is it important to know or follow? 

Practice! New improvisation environment needs practice to be felt comfortable and satisfying.

Otso:

The possibility to record Alejo brought a new level of listening and engagement to my playing. The session was very satisfactory and I felt it was the first time I experienced emotional engagement in music via web. 

I liked Alejo’s modular sounds, they were inspirational, well defined in their sound (freq. resonance filter sound sounds very consistent, strong, potent) 

Some clips – gain clipping or web bandwidth overflow? 

Alejo: 

Positive impression in general (since the beginning of the online sessions). 

I Like the material which is unexpected – processing unexpected material is different situation from processing one’s own sound. The other person’s unexpected sound drives the playing – and it’s not a machine – it’s a person. 

Very organic material gets generated. 

Very good listening situation at home – good level & monitor system.

Resonant filter as extra sound source.

(19) Web-mediated / four-hands modular I

———————————————————————————————————————————————

09.04.2020

Idea

Web-mediated four-hands modulation on a single signal path. Alejo modulates the synthesizer locally and Otso by the web via Max/UDP send. 

Hardward + software

Modular synthesizer

Sound source:

Manifold envelopes (6 envelopes) -> (c.v. input for control 6 harmonics’ level) -> Oscillator “Harmonic Oscillator” (harmonics master out)  -> 12-band fixed filterbank -> LPG -> delay -> sound interface -> computer -> Reaper -> Listento send/receive plug-in.

Otso - modulation: Korg midi controller -> computer -> Max (3 controllers midi data) -> UDP send -> internet -> UDP receive -> Max (3 controllers midi data) -> Expert Sleepers FH-2 (midi to cv) -> Manifold envelopes (v/8 Time) / Oscillator “Harmonic Oscillator” (v/oct cv in) / 12-band fixed filterbank (frequency scan center cv in).

Alejo - modulation: DJ controller XDJ-RX2 (midi data) -> computer -> Max (3 controllers midi data) -> Expert Sleepers FH-2 (midi to cv) -> delay (time cv in) / LPG (cv in) / random voltage generator “Wogglebug” (clock in). Random voltage generator “Wogglebug” (smooth out) -> Oscillator “Harmonic Oscillator” ( f.m cv in).

Otso and Alejo: Zoom -> web video call (audio and video).

Observations

General:

We now have a working situation with a complex network involving web, midi, modular synthesiser, audio/video streaming, two video cameras each, zoom software, Reaper and Listento send/receive plug-in.

Complex network involve different internet mediation tools: audio send/receive, ctrl message send/receive, speech send/receive, Video send/receive.

Controlling a rather simple synth.

Temporal dislocation needs to be regarded in detail! 

Spatial dislocation.

Web-mediation makes immediate understanding of what the other is doing harder to grasp – distanciation.

Otso: feeling of confusion with who’s doing what – the lack of visual / physical feedback makes it harder to navigate in the shared signal path. 

Alejo: Pleasant situation. Sometimes I felt that I wanted to move Otso’s filter frequency scan center cv in control, or that if I would be controlling that parameter, I would move it differently. Changes in the volume of each filter band also allowed me to contribute to its spectrum. It is a negotiation. Nice to play with a signal that has its own life!

(20) Web-mediated / four-hands modular II

————————————————————————————————————————————————

15.04.2020

Idea

Web-mediated four-hands modulation on a single signal path. Alejo modulates the synthesizer locally and Otso by the web via Max/UDP send. Izotope insight provides visual feedback on the signal.   

Hardward + software

Modular synthesizer

Sound source:

Manifold envelopes (6 envelopes) -> (c.v. input for control 6 harmonics’ level) -> Oscillator “Harmonic Oscillator” (harmonics master out)  -> 12-band fixed filterbank -> LPG -> delay -> sound interface -> computer -> Reaper -> Listento send/receive plug-in.

Otso - modulation: Korg midi controller -> computer -> Max (3 controllers midi data) -> UDP send -> internet -> UDP receive -> Max (3 controllers midi data) -> Expert Sleepers FH-2 (midi to cv) -> Manifold envelopes (v/8 Time) / Oscillator “Harmonic Oscillator” (v/oct cv in) / 12-band fixed filterbank (frequency scan center cv in).

Alejo - modulation: DJ controller XDJ-RX2 (midi data) -> computer -> Max (3 controllers midi data) -> Expert Sleepers FH-2 (midi to cv) -> delay (time cv in) / LPG (cv in) / random voltage generator “Wogglebug” (clock in). Random voltage generator “Wogglebug” (smooth out) -> Oscillator “Harmonic Oscillator” ( f.m cv in).

Otso and Alejo: Zoom -> web video call (audio and video).

Visual feedback:  Alejo - Reaper -> Master channel -> Izotope insight -> Zoom (shared screen).

Observations

Otso: Visual feedback proves to be very effective addition. The Izotope horizon/timeline is useful in creating patterns, rhythms, contrasts, gestures in spectrum and time. The web-latency is not a problem – I can correlate the audio and the Izotope visual feedback very well. It’s also nice to see Alejo – as a human presence. But I tend to concentrate on monitoring the signal visually and aurally – the combination helps in concentrating, focusing, getting into the sound. Opens the possibility to do less, listen to the other  

Alejo: spectrogram analysis is very detailed in izotope. Gives clear idea of how the other musician is interacting with the sound. I really like this addition.   

On the Internet experiments with Zoom I found important for performers to agree in what is important to include in the window in order to have good communication. 

For me the different latency of the videos is disturbing in a minor scale. It is somehow more pleasant to focus in listening and playing rather than seeing. The situations becomes a very acousmatic experience in a way. 

Having the possibility of seen the other performers' body and face expressions together with the spectrogram analysis and resent history feels also very important for shaping the communication between us.  

(21) Web-mediated / four-hands modular III

———————————————————————

16.04.2020

Idea

Web-mediated four-hands modulation on two signal path. Alejo modulates the synthesizer locally and Otso by the web via Max/UDP send. Izotope insight provides visual feedback on the signal.   

Hardward + software

Modular synthesizer

Sound source:

Manifold envelopes (6 envelopes) -> (c.v. input for control 6 harmonics’ level) -> 1st Oscillator “Harmonic Oscillator” (harmonics master out)  -> 12-band fixed filterbank -> 1st LPG -> 2nd LPG -> 1st delay -> sound interface -> computer -> Reaper -> Listento send/receive plug-in.

2nd oscillator “Telharmonic” (N, H and P) -> 1st VCA -> 2nd VCA -> sound interface -> computer -> Reaper -> Listento send/receive plug-in.

Otso - modulation: Korg midi controller -> computer -> Max (4 controllers midi data) -> UDP send -> internet -> UDP receive -> Max (4 controllers midi data) -> Expert Sleepers FH-2 (midi to cv) -> Manifold envelopes (v/8 Time) / 1st Oscillator “Harmonic Oscillator” (v/oct cv in) / 12-band fixed filterbank (frequency scan center cv in) / 2nd LPG (cv in) and 1st VCA (cv in). Manifold envelopes (mix out) -> 2nd oscillator “Telharmonic” (tonic).

Alejo - modulation: DJ controller XDJ-RX2 (midi data) -> computer -> Max (4 controllers midi data) -> Expert Sleepers FH-2 (midi to cv) -> 1st Oscillator “Harmonic Oscillator” (cv in) / 1st LPG (cv in) / 2nd VCA (cv in) / random voltage generator “Wogglebug” (clock in). Random voltage generator “Wogglebug” (smooth out) -> Oscillator “Harmonic Oscillator” ( f.m cv in).

Otso and Alejo: Zoom -> web video call (audio and video).

Visual feedback:  Alejo - Reaper -> Master channel -> Izotope insight -> Zoom (shared screen).

Observations

General:

Musical outcome might not be immediately good, but one can sense a potential (is it possible to specify the nature of the potential?).

Would there be ways to dig into the visual feedback? – individual channels, different representations?

Izotope shows a score, histo-spectrogram. Helps to build structure, connection & contrast. 

Seeing the body is important for macro event communication, mood, gaze, intentions and signs.