Sensory Cartographies

Ammassalik tactile maps from Greenland ca. 1880's

A research platform for spatial thinking, mapping and perception by Jonathan Reus and Sissel Marie Tonn

Both levels, intensity and qualification are directly embodied. Intensity is embodied in purely autonomic reactions most directly manifested in the skin - at the surface of the body, at its interface with things. Depth reactions belong more to the form/content (qualification) level, even though they also involve autonomic functions such as heartbeat and breathing.”

- Brian Massumi 'The Autonomy of Affect'

Of course, a musician can also alter the affordances of her environment, but she is simultaneously dependent upon the situation (Gibson 1986: 143). I could call this mutual determination and interdependency “resonance,” the active exploratory engagement and interaction between organism and environment (Clarke 2005: 19).

Points of Departure

Anthropologist Judith van der Elst:

  • spatial thinking is not universal across populations.
  • Differences in space-time conceptualisation affect the way we navigate, design, interact with, and understand the world around us.
  • The development of spatial information systems can provide new ways of exploring this diversity in human thinking, challenging the ‘view from above’, towards a more ecological view on human perception.

Arakawa & Gin's Reversible Destiny Architecture

Background Work

Sissel's artistic practice is based on interests in ecology, the body and sensing and environmental change. Jon comes from a background in music, instrument building and critical media scholarship. He had previously started building an instrument that used gsr signals as a way of improvising between two performing bodies. 
 
We wanted to create a project that looked critically at instrumentation and measurement, but that also investigated how we perceive an environment.

Micronesian / Marshall Islands Stick Map - Subjective mapping of environments of flux\

Autonomic Resonance - Jonathan Reus and Momoko Noguchi. Free improvisation with hand-made biometric recording devices, visual projections, and musical artificial intelligence.

Inspiration

A fascinating aspect of the charts is that they varied so much in form and interpretation that they were “readable” only by the specific navigators who constructed them. Evidently the knowledge contained in each was a closely guarded secret.

Sensory Cartographies is an ongoing research project conceiving of new mapping technologies and practices that take into consideration the constant flux that plays out at the interface between body and world. We are experimenting with and contemplating the way mapping technologies determine the way we experience space.

Mozzi Library
(by the great efforts of Tim Barrass & team)

Rebecca Horn's Body Extensions

earthquake vest

sensor & actuator module

Lygia Clark's Relational Objects

The instruments are inspired by the body’s ability to act as a filter, singling out and amplifying events in a sea of impressions.
 
Perceiving the environment as a sea of impressions, we were inspired by how the brain filters out information in order to make sense of the world (sensory gating).
 
We are culturally trained to be more sensitive to some events than others. The instruments question or challenge how we are singling out certain events, such as the direction of the wind, the difference in sight and sound between fog and sunlight, subtle altitude gradients of the earth, and recast these experiences well beyond the threshold of detection.
 
The instruments amplify specific subtle impressions of the environment, filter the natural environment into certain frequencies, and extract and remap flows that are ephemeral and below the just noticeable difference of our awareness.

Platform Development

The Intimate Earthquake Archive

 - a project by Sissel Marie Tonn, in collaboration with Jonathan Reus and Carsten Tonn-Petersen.

- A wearable-enabled tactile 'archive' of the drilling-induced earthquakes in Groningen, NL. 

Bela

Augmented Instruments Laboratory (Queen Mary)

http://bela.io/

 

Sensory gating describes neurological processes of filtering out redundant or unnecessary stimuli in the brain from all possible environmental stimuli. Also referred to as gating or filtering, sensory gating prevents an overload of irrelevant information in the higher cortical centers of the brain
(- wikipedia)

Luke Howard's taxonomy of Clouds

Let’s just keep it spread out on the empire,” suggests a young map-maker, “it’s a perfect fit.” So they do, and now all the people go about their business walking on the map like a giant carpet. Time passes. The people forget they are walking on the map. The empire changes. Rivers shift; mountains rise, others erode."

- Glazebrook & Conrad

Think of ‘mapping’ as the flux of relations between body and environment

Ernst Haeckel and assistant in full explorer gear

'External' Sensory Organs

It is never filiations which are important, but alliances, alloys; these are not successions, lines of descent, but contagions, epidemics, the wind.

An animal is defined less by its genus,
its species, its organs, and its functions, than by the assemblages into which it enters.

Deleuze and Parnet 1987: 69

"We started out exploring the relationships between consciousness, body and environment, as mediated by the filtration mechanisms of the senses, by creating worn instruments and exercises, which help us rethink modes of collecting, categorizing and mapping from the perspective of the lived experience of a place."

"We use external objects as mental tools naturally and compulsively. Our minds are plastic and extensible, so fundamentally embodied that the barrier between world and mind is leaky like a sieve."


- Tom Stafford in Tools for Thought

"Systems of classification, as much as sextants and chronometers, allowed Europeans to perceive themselves as the magistrates of Providence, equipped by their knowledge of its laws with responsibilities over all of creation."


- Richard Drayton, Nature's Government

Reaching Inward

Measurement of Psychobiological Activity

The autonomic nervous system

In 2013 Sissel visited the herbarium/seedbank and botanical garden, and was fascinated in the colonial past and how it was present at the Island. She started researching the ‘collection mania’ of the Age of Exploration, and how the island showed all the traces of botanical exchange.
 
Madeira is the reification of both a logico-scientific worldview (that the natural world can be compartmentalized, classified and made into a ‘standing resource’) as well as of colonial expansion. The connection between colonial expansion and scientific ‘classification impulse’ are entangled in the idea of the ‘standing resource’.

"Thus the living thing has an exterior milieu of materials, an interior milieu of composing elements and composed substances, an intermediary milieu of membranes and limits, and an annexed milieu of energy sources and actions perceptions."


Deleuze and Guattari 1987: 313

"By looking at your arousal as you traverse physical space you create a medium for reflection on your feelings, feelings which can normally pass below the radar of conscious awareness. Similarly, on a group level, when we can look at similarities in our emotional responses to physical spaces it lets us focus on shared feelings which, are normally only fleeting and not recorded, and so pass below the collective radar"

- Tom Stafford

Inner Movements Workshop

Workshop and Urban intervention in Spektrum, Berlin

Herbarium collection,

Jardim Botanico, Funchal, Madeira

The Electro-dermal Response

 

An exosomatic method of measuring the body's autonomic response to external & internal events.

 

 

Electrodermal Activity (EDA) or Galvanic Skin Response (GSR) measures the level of autonomic system activity by measuring the electrical resistance of the tissue path between two electrodes applied to the skin. This technique has been extensively used in animal and human research on pain, anxiety and stress levels and it has been a key part of lie detection. This article will examine the history of EDA, review significant research applicable to laboratory animals, and suggest possible applications for EDA technology in routine laboratory use.


Jardim Botanico, Funchal, Madeira

Ecological Thought:
For Gregory Bateson the world is perceived through movement of the subject. The body doesn’t stop at the skin. Information only exists thanks to the movement of the perceiver relative to his or her surroundings. 
 
he states: stable features of the world remain imperceptible unless we move in relation to them. Through ‘a scanning movement’ (the blind man swinging his cane) we draw distinctions, in the sense of not representing them graphically but of ‘pulling them out’.

Music as I conceive it is ecological. You could go further and say that it is ecology.
(Cage 1981: 229)

Charles Féré (1888) - in the lab of Jean Charcot - found that, by passing a small electrical current across two electrodes placed on the surface of the skin, one could measure momentary decreases in skin resistance in response to a variety of stimuli (visual, auditory, gustatory, olfactory, etc.). The basic phenomenon discovered by Fere is that the skin momentarily becomes a better conductor of electricity when external stimuli are presented.

  • tonic level The absolute level of resistance/conductance at a given moment in the absence of a measurable phasic response. This is the SRL/SCL (skin resistance/conductance level).

  • phasic response these are dynamic decreases in resistance referred to as SRR/SCR (skin resistance/conductance responses).

“Knowledge does not keep any better than fish.”
- Alfred North Whitehead

“Some aspects of our lives are more comparable to fungal indeterminacy (…). Our daily habits are repetitive, but they are also open-ended, responding to opportunity and encounter. What if our indeterminate life form was not the shape of our bodies, but rather the shape of our motions over time? “

 

- Anna Tsing, The Mushroom at the End of the World

Rosalind Picard "Affective Computing" http://web.media.mit.edu/~picard/

Affective Computing & Self Quantification

Carl Jung

Studies in Word Analysis (1907)

 

The instruments challenge how we are singling out certain events and connect us to events outside of direct control, such as the direction of the wind, the difference in sight and sound between fog and sunlight, subtle altitude gradients of the earth, and inner movements of psychobiology. They are drawing new maps of sensation.

"all our tools are vehicles for self-awareness. Even physical tools like a hammer. As soon as you pick it up you ask yourself “What can I hit with this?”, which is encouraging us to ask ourselves “What do I want to make or break with this?”.

- Tom Stafford in Tools for Thought

"Drawing upon inuit observations that ‘the world has titled on its axis’ argues for the re-ordering of our biological perception under the anthroposcene: “ it appears that Husserl’s idea of the earth as ‘original ark’ is now obsolete; we now have to recalibrate our sensorial systems to adjust to contradiction, catastrophe, and ecological volatility born of human activities that override and neutralize long-standing histories of local knowledge." 


- Amanda Boetzkes in the anthology Art in the Anthropocene

“Where technology and nature was traditionally seen as opposed, they now appear to merge or even trade places. We must no longer see ourselves as the anti-natural species that merely threatens and eliminate nature, but rather as catalysts of evolution. - we must learn to think transversally.”

- Eric Kluitenberg, Techno-ecologies

EDR electrode placement

for psychiatric research

Collective Performance: Exploring neural plasticity through Peripheral Vision exercises

Faith, Beliefs, Myths and Propaganda


Biometric Archiving

- Exhibition of biometric data, sensation-collection tools and video material from the Laurisilva Forest.

Jardim Botanico, Funchal Madeira

"Technologies present themselves as the forward image of our desires, and these forward movements often keep us from sensing legitimate disappointment in them. The technological has always promised a great number of apocryphal benefits: from truth telling, to bodily enhancement, to cognitive amplification."

- Jamie Allen

 

 

"Often things that don’t work simply must, as we’ve invested so much time, effort, emotion, belief and money in them. This rather legitimate disappointment is something we hide from ourselves and each other, making the technologies ‘apocryphal’: dubiously authentic, reliable, and functionally suspect. All thinking is speculative, and technologies absorb and condense this speculation, in particular as it intersects with ‘spiritual’ practices."

- Jamie Allen

David Rosenboom, John Lennon, Yoko Ono sonic biofeedback on the Mike Douglas Show (1972)

Apocryphal Technologies


Performing Sensory Cartographies

- Artist workshop and collaborative development of performance in Zone2source, Amstelpark, Amsterdam

Scientology E-meter

The resistance of a dead female body is 5,000 ohms and of a dead male body,12,500 ohms.

L. Ron Hubbard

Bad Practice Taxidermy?

The Incredible Montage

from The Parallax View (1974)

Dick Raaijmakers

The Graphic Method Bicycle (1979)

Notes for Peripheral Vision Performance: 

 

Scanning the room: Vision implies linearity, Meeting a person: locking eyes. Being at war:  the 'fate line' of shooting someone. There is a direct relation between war and vision. 

The evolved brain as filtration mechanism for attention. Making sense in sea of relation to avoid sensory overload. 

How we sense naturally have political implications. Gaze, but also hierarchy of attention, of the senses and the speed and care with which we focus our attention. 

Being a baby: We are born helpless (zebras can run 45 mins after they are born), we learn on the job, we are plastic, we can adapt to different environments. We exist in a constant interplay with everything around us, and we are in no way static beings. We get moulded to fit the world around us.

We need to change our concepts: This entire year has been the hottest on record. We are not hard-wired to perceive environmental change. A common rule in adaptation is that if you cannot perceive a threat you cannot act upon it. Perhaps we need to re-consider our span of attention. Perhaps it is time to challenge preconceived notions of how we perceive change within our environment. We are culturally trained to be more sensitive to some events than others. 

Ferrets hearing colour: When the default operations of the brain is challenged we understand better the processes with which we function. These goggles does not rearrange my neurons directly, like some MIT researchers did by rewiring the auditory and visual cortex of ferrets, but they challenge my focal line. They disrupt my bodily experience making me question what I pay attention to. 

They are also a technology (although cheaply made from cardboard and some broken-up makeup mirrors). The technologies we create also enact a highly subjective selection of how we perceive our world, and as they extend our bodies they condition how we move, live, remember. 

Peripheral vision exercise: We can all train our peripheral vision by holding out our fingers, focusing not on the person before us, but on the dancing movements of the surrounding landscapes, of the colours and shades, shapes and sounds around us.

What Is It Like To Be A Bat?

- Exhibition of biometric data, sensation-collection tools and video material from the Laurisilva Forest.

Zone2Source, Orangerie, Amsterdam

Of all psychophysiological variables, the GSR can lay reasonable claim to being the most popular in current use. In spite of years of searching study, we are still surprisingly uncertain about the function, not to say the mechanism of this phenomenon. Nevertheless the GSR seems to be a robust sort of variable since, in hundreds of experiments, it continues stoutly to provide useful data in spite of being frequently abused by measurement techniques which range from the arbitrary to the positively weird.

- Lykken & Venables 1971

Alvin Lucier

Music for Solo Performer (1965)

Methods for Deep Worlding

Augmented Attention Lab


Collaborative pressure-cooker focused on prototyping new technologies, gestures and practices, which challenge the ways in which we pay attention. The Augmented Attention Lab #1 was a part of Sensorium Festival in Bratislava, Slovakia in 2019.


“Our minds are plastic and extensible, so fundamentally embodied that the barrier between world and mind is leaky like a sieve”.


- Tom Stafford in Tools for Thought

A Sensory Cartography of the Wadden Sea

"Progress is a forward march, drawing other kinds of time into its rhythms. Without that driving beat we might notice other temporal patterns. Each living thing remakes the world through seasonal pulses of growth, lifetime reproductive patterns.. Within a given species too there are multiple time-making projects, as organisms enlist each other and coordinate in making landscapes. The curiosity I advocate follows such multiple temporalities." (Anna Tsing, The Mushroom At The End Of The World, pp.21)


What each generation contributes to the next, in this process, is an education of attention (Gibson 1979: 254).


 

"Rather than merely incorporating the Other as a map subject, we should think more deeply about Othering cartographic subjectivity, or acknowledging that Others have developed their own map-making practices that diverge from Western convention."
 
Shannon Mattern, "Mapping's Intelligent Agents", Places Journal 2017

 

Research tool: Deep Worlding

A major goal of this project is the development of methods to support the process of Deep Worlding in response to specific sites and communities. These include everything from anthropological methods for gathering stories to computational data visualization in 2D and 3D. Our goal is to be able to create unique aesthetic “maps”; through experiences and activities that trigger new awarenesses of place for individuals and groups.

Proposal for the creation of "Remote Sensing Organs" for Sensorium Festival 2020.

Performers transmit their biometric data of experiencing the changing tides of the Wadden sea via internet uplink to "organs" in the exhibition space in Pistori Palace, Bratislava.

On the temporality of starfish moving across a seabed:


"The lens of a camera—capable of capturing increments of change at a
different number of frames per second than the human eye—floods the stillness
of such an underwater scene with innumerable details, revealing contracting arrays of orange dots and spines moving with hydraulic slowness as individual tube feet attach and detach from the rocky substrate. Such a mediated series of images brings the rhythms of starfish time into contact with the temporal pace of the human perceptual world."

- Ada Smailbegović in Cloud Writing: Describing Soft Architectures of Change
in the Anthropocene

The task of each student will be to develop a sensitivity to their own route, and to the mediating technologies they have brought with them to document their experience. Some questions to ask during the walk:

 

  • What do you pay attention to within this space?

  • Is there a pattern to the way this environment, and your recording media, demand your attention? Can this pattern also be annotated, and used as a method for moving and recording?

  • What senses are prioritized in your experience?

 

Rather than producing efficient or objective maps for A to B navigation, this exercise is a way to cultivate an embodied awareness of how senses and recording media privilege certain relationships to navigating the environment over others, and to open a space for subjective navigational strategies to emerge.

Model of mussel beds from Wadden Sea by NIOZ researcher Johan van de Koppel.

 

"Non-human mapping agents — from Lidar sensors to drones to plotting robots — can be critical agents in the cartographic enterprise. Yet the non-human Other is also an agent in those landscapes we seek to map. As we’ve already seen, fish and waves dominate Pacific Islanders’ seascape epistemologies; ice and hunting paths shape Inuit maps; and flora and fauna are central to the landscape imaginaries of environmentalists and extraction companies alike. In most Western frameworks, non-human Others, from animals to plants to minerals, are represented as resources to be exploited: things to eat or mine or look at in a national park. But now the convergence of certain new ways of thinking — Anthropocene studies, new relational ontologies, Latourian actor-networks, feminist ethics, and what Donna Haraway describes as “SF” thinking, science fiction, speculative fabulation, string figures, speculative feminism, science fact, so far — has inspired many contemporary mapmakers to think about how to represent these Others not merely as human resources, but as entities with equal claim to the landscapes they inhabit, and with their own spatial intelligences."

 
Shannon Mattern, "Mapping's Intelligent Agents", Places Journal 2017

 

Click to see students' Research Catalogue!

Sketch for "Memory Spaces": We will conduct interviews and document local knowledge of the land. We organize expeditions where local residents bring us to significant areas that they believe are undergoing change, and use a combination of biometric data gathering exercises and ethnographic interview methods to document how they experience these locations. With participants we will gather geotagged biometric data and topographic data in the field, which will later be used in the creation of three dimensional “memory maps”, physical objects that can be carried as a psychological link between the wearer and a time, space, or place. We want to use the remainder of our residency time exploring techniques for creating different forms of memory maps such as data-driven parametric design approaches (using Blender and 3D printing).