Nonrepresentational Practice and Intersemiotic: Annotations about Rhizomatic Thinking in In Vitro, An Installation for Camera Obscura
"The listener is also a soundmaker, and even the sound of one's own voice comes back to the ear colored by the environment."
(Barry Truax, 2001)
Image 6: Day transitions in the camera obscura. Photographs by author.
The Camera Obscura shows us live the actual happenings of the Plaza San Mateo surroundings in one of the walls inside the building. It starts to appear with the first rays of light and stays throughout the day, disappearing with the sunset. At first glance, this projection barely shows what we hear of the outside due to its upside-down reflection, enlightening mainly the top of the buildings, that are static during all presentations of the image. However, this “top of the buildings” does not represent a simple element. It is instead a complex figure drawn on the frame of the camera obscura, and it undergoes visually perceptible development throughout the day. During the days when the installation was planned, the camera obscura was visited in different moments during the day. One can perceive the different textures of this top of the buildings and the relation of these kinds of textures with the amount of sunlight and clouds. On a typical Bucaramanga sunny day, there is a constant twelve hours of daylight, increasing in a linear form and reaching climax mostly at midday. The light increases for six hours after the sunrise and decreases for another six hours until sunset. This slow gradation of light, imperceptible for a viewer, reminds one of Truax’s ocean of sounds, alluding to the background of a soundscape that is more than a background. It represents a large amount of information that interacts with other elements, but that representation stays out of our attention while we listen to the soundscape or, in this case, appreciate the landscape.
The relationship between this ocean and other elements can be perceived in the clouds. On a sunny day with a clear blue sky, the texture, and details of the top of the buildings are more visible, while on a cloudy day there is less light and therefore less texture. It is very noticeable how the changing proportions of light throughout the day lead to interchanging roles of leadership between the top of the buildings and the sky. Even so, what we considered a “protagonist” role on the recording such as the claxon of the soundscape, is missing so far on the image.
In comparison with the soundscape analysis and its “protagonist” role represented on the recording by the claxons, the element that stands out and shines, highlighted from the others are the flying birds. The active participation of the birds in the landscape that can be seen on the camera obscura is significant due to the shadow produced when a bird starts to fly from the top of the buildings or when arrives. Notwithstanding, this active participation is in ongoing dialogue with the behavior of sunlight. It is necessary to have a sunny condition to better appreciate this “protagonist” of the landscape. In that sense, and remembering the enlightened points of the soundscape composition, and the model of analysis that, among other things, proposes the idea of multiple roles of participation instead as individual entities, the flying birds’ could be interpreted as a player that interacts with the environment, remarking upon the landscape, appearing and disappearing with the amount of light, and also a part of this environment that is present under the correct conditions. To rephrase it, the flying birds could be interpreted as an independent element or as a characteristic of the background of the sky that is remarked only when the light situation allows.
REVEALING THE INVISIBLE IN THE IMAGE AND THE INAUDIBLE IN THE SOUNDSCAPE
Image 4: Interconnection between three tubers in a rhizomatic form. Illustration by Zulma Rueda
To start, I considered the fact that the three elements involved in the artwork have a structure, can be read, and can be interpreted in multiple forms. In other words, the soundscape, the camera obscura, and the building can be interpreted as texts. The intersemiotic relation between the soundscape and the image projected by the camera obscura therefore began with the recognition of those elements as two texts of different nature. That recognition implies understanding each element as a semiotic mode that communicates and has a structure and meaning of its own. The next step was to establish an equal model that could approach those two different nature texts.
Thinking through Deleuze and Guattari’s rhizome principle, I then decided to choose a model that could analyze images and sound, and that could then translate those analyses into the other semiotic mode. The model chosen to analyze sound and image was the one proposed by Barry Truax (2001) to analyze what he calls “systems of acoustic communication,” especially thought for sound analysis. For Truax, the systems of acoustic communication refer to all manifestations of sound, ordering from the simplest to the most complex sound manifestation in a continuum. Truax refers specifically to three “major” systems: Speech, Music, and Soundscape. Our sound is located in the major system soundscape. The analysis model proposed by Truax (2001) is based on the structure of the system of communication as the mediator between sound and meaning. In the case of soundscape, the structure is open to its capability to give significance to a separate sound and to construct relationships with the environment “and the context within which all of it occurs” (Truax, 2001, p.48).
Diagram 1: Model of analysis of sound. Based on Truax (2001)
For the projection of the camera obscura analysis, I took into account that the image's structure was akin to that of a landscape.. In relation with Truax’s model of analysis of acoustic system of communication, a landscape’s structure as the mediator between image and meaning is based in the contextualization of environmental and individual visual elements in a descriptive or ontogenic manner.
Diagram 2: Model of analysis of image adjusted. Based on Truax (2001)
The result of analyzing those two texts from an equal analysis model permits one to make connections between the results and to integrate them in an intertext that can be analyzed with Truax’s system of acoustic communication model.
To connect the space, I considered the fact that even though a building can be interpreted as a text with all of what it represents in terms of analysis (in contrast to the camera obscura and the soundscape which moves in time and can be analyzed in an ontogenic manner), a building cannot be interpreted by its development during the day, as it is a static structure that remains constant over time. For that reason, the approach to the space to make connections with the other texts is from another perspective than the intersemiotic. For that, NRT was the chosen alternative to analyze the building where the embodied experience will take place. At the same time, relating the location with NRT proposes strategies to think about nonrepresentational structure and poses the materiality and the production of multiple subjectivities as pillars of the experience.
SOUNDSCAPE AS SYSTEMS OF ACOUSTIC COMMUNICATION
To make the urban soundscape composition, I selected as the main sample a recording made on a Monday morning outside the building, one of the noisiest places in Bucaramanga. In this recording, it is possible to hear many sounds that can be categorized as human sounds (voices and steps), and mechanical nonhuman sounds (cars, public transport, and some machines of the market stores). Both categories are the result of the many activities that people undertake during the day in the zone. However, during the whole sample of soundscape recording, the first thing that can be identified as a potential meaningful sound is the background, which itself is composed of a numerous individual sounds. Truax (2001) refers to this complex sound that we interpret as a single background sound of a soundscape as an ocean of sounds, also proposing a process he calls profound listening as the type of listening that takes place when some sound stays out of our primary attention (cited in Secco, 2017, p.10).
First, the Plaza San Mateo was the location selected. This Historic building, which was the central marketplace in the 1970s, is located in the main center of Bucaramanga and is surrounded by the most central district of the city, where most of the working class, shop, meet, and work. The Plaza San Mateo was built in the early 20th century to give the community a better marketplace and was abandoned in 1979 after a big fire that left many material losses.1 For the occasion, this location was entirely reimagined by nine artists, not as a gallery but as polyvalent points inside the building with a central guiding concept: make this abandoned building a public space again.
Image 1: Inside the Plaza San Mateo
The composition of a soundscape was the second principal element, and it was conceived in relation to the first one (the location). The soundscape composition was based on the urban soundscape of Plaza San Mateo surroundings in a normal day. It is important to highlight that after the fire that left Plaza San Mateo empty and useless, all that we could associate as the representational soundscape that would have previously occurred inside the building moved to the outside on the streets. The soundscape of the surrounding neighborhood on a normal day is a cluster of millions of individual sounds that confluent systematically, making multiple representations and narratives in the same intervals of time.
NONREPRESENTATIONAL THEORY IN ARTS
Artistic practice is a predominant mode of research through experimentation between different semiotic modes. It offers a procedure that takes the relation of the artist with the matter (artistic physical material) to another level, sometimes as a co-creator role. Nonrepresentational theory (NRT), related to poststructuralist philosophy (including Deleuze, among other authors) offers an epistemology of multiple methods of approaching the act of thinking from practice and the “embodied experience” (Thrift 1996, 2000).
For the theorists of NRT, representationalism has the opposite effect of what is intended, rather than creating unity and coherence, representations are reduced to the act of framing, which leads to a sense of separation and disconnection (Williams 2020). That is why NRT moves between interpretation, mediation of interpretation, and the crafting of representations, moving away from the purported meaning of representation. In addition, it proposes in its conceptual pillars the production of multiplicities of subjectivity; enlightening the materiality, affect, and embodiment as fundamental roles to constitute subjectivity (Williams, 2000). As an example of how the NRT is present in artistic practice, Robert Kruse (2019) proposes how to interpret John Cage’s music via NRT, suggesting that in works such as 4’33’’ Cage achieves a nonrepresentational experience for the audience through his conceptual proposal to approach the space and its relation with music and performance (Kruse 2019).
Intertextuality, transtextuality, or intersemiotics translations?
It is well known that there is no concrete concept of intertextuality. Several authors such as Roland Barthes or Julia Kristeva propose a general meaning of intertextuality. For Roland Barthes every text is a “camera of echoes” (Barthes,1978 p.84), proposing that every text is an intertext that cannot exist without other previous texts. For Kristeva (1969) “each word (text) is an intersection of words (texts) where at least one other word (text) can be read. Any text is constructed as a mosaic of quotations; any text is the absorption and transformation of another” (Barthes, 1978 p.85). As we can notice, the definitions proposed by Barthes and Kristeva are similar and mainly focus on traditional studies of textual analysis (e.g. literary analysis). But, what about different textual manifestations? Is there a possibility to have an intertext between two images or two songs?
To answer this, it is important to refer to other authors such as Gerard Genette (1997). He redefines the idea of intertextuality as transtextuality, an effective presence of one text in another (by a quotation or an allusion). Nevertheless, although the idea of what a text is has changed for contemporary theorists (see for example Bakhtin, Riffaterre, Gignoux, Piégay-Gros), for Genette (1997) transtextuality occurs with two “texts” of the same nature. However, is it possible to have an intertext between two different texts of different nature?
I want to propose in this text that this movement from intertextuality to intersemiotic meaning is better understood through the rhizomatic thought of Deleuze and Guattari, which imagines experimentation and interconnections between knowledge as new paths on a map: “The rhizome is altogether different, a map and not a tracing. Make a map, not a tracing. The orchid does not reproduce the tracing of the wasp; it forms a map with the wasp, in a rhizome” (Deleuze and Guattari, 1978, p.12).
METHODOLOGY
To fully comprehend the rationale behind the decisions made for the soundscape composition, it is crucial to reiterate the primary objective of our program - to reveal the space. The soundscape composition was conceived as a means of bringing together the diverse materials gathered from the surrounding soundscape, the camera obscura, and the building itself. The challenge was not simply to create a soundscape from the sounds of the urban environment, but to integrate sound, live-action imagery, and the physical space of the building in a way that would effectively reveal the space. This required a thoughtful and innovative approach, which resulted in a composition that was both unique and effective in achieving our objective.
“In Vitro” is a multimedia work part of a bigger multidisciplinary work based inside an abandoned patrimonial building. This installation is composed of three principal elements: The location, a soundscape composition, and a camera obscura.
Image 2: The landscape of the Plaza San Mateo surrounding © Fernanda Niño Díaz
Soundscape recording of Plaza San Mateo surrounding
Third, a camera obscura is the last principal element that comprised In Vitro. This one is also directly related to the location and the soundscape composition. Due to the ongoing decay of this building, a small part of the facade fell, letting the perfect amount of light pass and suddenly become a camera obscura. The resulting phenomenon shows live a landscape of the exterior and what is happening in the surroundings, including the market activities that used to be inside. All of this is projected inside the place where the installation was planned, while barely listening to the exterior.
On the other hand, some sounds emerge in a protagonist role and enlighten punctual points of the spectrum during the recording. Examples of this are the horns of the vehicles that acoustically stick out of this ocean of sounds, appearing chaotic and untimely during all recordings. According to Truax (2001), in what we understand as human experience in this world, all manifestations of sound are forms of communication. His main idea of “acoustic communication” suggests an alternative approach to the behavior of sound and its relation within the listener, sound-maker, and the environment, proposing roles of participation, not as individual entities but as polyvalent players. In that sense, for Truax, the enlightened points of the soundscape recording could be interpreted as a player that is interacting with the environment and as a listener and a sound maker as well. In other words, the horns represent the traffic heard in the background that is interacting with people on the streets, increasing and decreasing in direct relation from people’s sounds, heard in the background as well. A soundscape like the Plaza San Mateo on a Monday morning recording is a sound system that intercommunicates itself, as a natural music improvisation that involves natural and non-natural participants. Nonetheless, the urban soundscape of the Plaza San Mateo location cannot represent the entire landscape of this overcrowded point.
In 2018, Espacios Revelados - Changing Places, a festival(?) organized by Siemens Stiftung,2 took place in my home city of Bucaramanga. The event recruited artists and commissioned artistic works that highlight the cohesion between the social context of a city passing through a fast social transformation and its realities in terms of urban places. The program is focused on Latin American countries and had previously been held in other capital cities such as Santiago de Chile and Buenos Aires. It was attracted to Bucaramanga, a middle capital city in Colombia, because of its fast-changing situation due to growing migration, rapid economic environment, as well as the possibility to work with a public cultural institution as a host to operate the program, ensuring the complete diffusion of the calls and the participation of the artistic community.
As part of this program, I had the opportunity to co-create a work in a big abandoned patrimonial building, working together with plastic artists to turn an old market square into a public space again.
In this text, I present some decisions about how I chose to define the symbolic and conceptual connections between sound, visual material, and space within this project. I outline my approach in developing the sound material for “In Vitro,” the sound installation part of the larger work, titled Ágora. I present the origin of the idea and the decisions made, considering the relations between the space inside and outside, the context of an old public place, and the framework of revealing a public space as a nonrepresentational practice.
This text concludes by posing two reflections that emerged during the development of the work. First, Gilles Deleuze and Felix Guattari’s (1972) rhizomatic thought offered an effective tool to understand sound, image, and space as three different intertexts. This enabled me to analyze them as communication systems, a concept proposed by Barry Truax (2001), allowing one to catch exogenous meanings between the three and realize direct relations between concepts such as Truax’s soundscape composition analysis and intersemiotic relation between image and sound.Second, this work proposes to approach the conceptual objective of the program “revealing a space” through Nigel Thrift’s Nonrepresentational theory (2007) to focus the concept of “public” as the main nonrepresentational meaning on the artistic practice.
In Vitro
Image 3: Inside Plaza San Mateo
The integration of these three elements is based on the idea of "revealing the space" and creating connections between them. The goal is to highlight what may be missing in each semiotic mode while also allowing them to interact and complement each other. This approach emphasizes the importance of integration and collaboration in creating a cohesive and meaningful whole.
The guiding framework of “revealing” proposes a new modality through which to live the urban landscape of Bucaramanga and more specifically the Plaza San Mateo surroundings. For that, “In Vitro” is made using the conventional urban soundscape of the Plaza San Mateo and through signal processing which transforms and recreates a parallel version of the outside sound reality. This electroacoustic echo of the outside was accompanied by the real-time context of the exterior space via the camera obscura image.
It is important to remark that I thought this artistic work as a nonrepresentational experience, avoiding any representational structure or way to listen. Instead, I presented it as a transdisciplinary work that invites the audience to reconceptualize and to trace a path from subjectivity and into an embodied experience. To reinforce those pillars of the nonrepresentational theory, and taking into account that the live projection of the camera obscura was always present during the daylight, I made a soundscape composition of 300 minutes of length that was composed without signposts that could suggest a structure, beginning or ending, allowing it to repeat and loop if it was necessary.
In the next section of this text, I present the concepts of rhizome, nonrepresentational theory, and intersemiotic relations to expound how I integrate those three different semiotic modes, while recognizing those as texts with different natures. For this, it is important to highlight the direct connections and circumstances related to the proper materials of each manifestation to explain the decisions made for the signal processing of the urban soundscape of Plaza San Mateo.
RHIZOME AS A PARADIGM TO UNDERSTAND THE CREATIVE PROCESS
Deleuze and Guattari’s concept of rhizome (1972) emerges in opposition to the act of heirarchical classification as a form of determining concepts and knowledge. It is usual to conceive types of classification in scientific fields such as the arborescent classification in biology, very common in taxonomy or evolution diagrams, proposing a hierarchical system to organize concepts, and having a procedural method to receive information from a higher unit of the preestablished path. For Deleuze and Guattari, however, in philosophy and human sciences to determine a classification denies other possibilities to make connections and to appreciate other forms of knowledge and understanding. In their words:
The rhizome pertains to a map that must be produced, constructed, a map that is always detachable, connectable, reversible, modifiable, and has multiple entryways and exits and its own lines of flight. It is tracings that must be put on the map, not the opposite.
The rhizome opens multiple forms and routes to connect points in-between, creating power organizations and semiotic relations between ideas and abstractions in a nonhierarchical rhizomatic way. For that, Deleuze and Guattari use the term “tuber” as a metaphor for the rhizome root in comparison with the arborescent root. In that sense, a semiotic relation is a tuber that grows and is fed by diverse interpretations and a cumulus of ways of approaching the relation. Considering rhizomatic thinking, among other methods, as a way to trace a path without denying other possible trajectories, the act of artistic practice can be understood as an effective application of Deleuze’s and Guattari’s rhizome principle of connection and heterogeneity: any point of a rhizome can and must be connected to any other (1972).
Image 9: Sound analysis based on the McAulay-Quatieri with the ocean dried.
The next step was to transpose that “residue” of the dredged ocean and superimpose it on the protagonists that were all between 20 and 2500 Hz. The result was a transformed soundscape that, indeed, reminds the original soundscape recording of the Plaza San Mateo Surroundings, but evokes a sort of ghost new sounds that paint with new colors and participate as active elements in this intervened recording.
Image 7: The landscape of the Plaza San Mateo surrounding and the camera obscura. Photographs by Fernanda Diaz Niño and the author
Zábala (2008) proposes an intersemiotic relation between two or more semiotic systems, first to know the formal components of the systems involved. In our case, and basing this work mainly on the multiplicity principle of the rhizome two elements of the piece were analyzed from Truax’s (2001) acoustic communication concept, understanding that the soundscape and the camera obscura participate in an integrated landscape of Plaza San Mateo, sharing the same role as polyvalent players. The camera obscura has the flying birds as the equivalent of “protagonist” elements that are highlighted mainly by their movements. This dynamism present on the wall as a screen could be translated as a potential leitmotif in music, or a memory-generator element. These characteristic elements can also be identified on the soundscape as the horns of the cars, as in this case also the element of the horn has the same protagonist potential. Additionally, the ocean of sounds comprised of the thousands of individual sounds of daily noise has an analogue in camera obscura’s “top of the buildings” element, both of which show a continuum that progresses during the daylight and has a climax point at midday when the soundscape is in its densest state. In this view, we have two types of protagonists and two oceans as well. However, despite the direct connection between both elements the ocean of sounds and the “top of the buildings” the protagonist elements of the “flying birds” and the horns of the traffic do not have a systematic connection.
That is how the decisions that were considered as a material for the soundscape composition started. The idea of revealing the space must be focused on showing as much major information about the place as possible. Therefore, thinking the installation as a whole that is covering sound and image, it was considered relevant remarking all the existent “protagonist” in the soundscape composition, as some of these we cannot appreciate on the camera obscura, but in the spectrum of the sound.
Image 8: Sound analysis based on the McAulay-Quatieri Technique with the protagonist selected on the sound analysis
For that, and taking into account the ways that those remarkable sounds “navigate” this ocean of sound, my strategy to bring them to the forefront was to “dredge” all of the soundscape partials that could enter on the “easily perceptible” label, leaving just the protagonist elements and their residue.
Image 10: Sound analysis based on the McAulay-Quatieri with the transposed residue
It is this new processed recording with its foregrounded materials the brute sound material that was used to compose the soundscape. As there are already full of connections between camera obscura and the soundscape in this sound material, most of the composition decisions were made using personal perception and experimentation as parameters to determine the amplitude changes and transposition of the frequencies.
MAKING A PUBLIC SPACE FROM NONREPRESENTATIONAL PRACTICES
Finally, the historic building of Plaza San Mateo is a transcendent element of the work. It is important to mention that, due to its abandoned state, the process of composing the idea, recording the soundscape, and presenting the work itself were always conditioned by permissions, risks of collapse, and bureaucratic procedures. The possibilities of what kind of work we could create in the space were so concrete and limited, converting the space not just the location but also an element that interacts, conditioning some decisions of the artistic work. Conceiving then, that the space must remain be as intact as it began, and using the entangled principle of heterogeneity, the relation between the space, the camera obscura, and the soundscape compositions were made by the audience through a nonrepresentational practice. Materiality and nonhuman agency principles became the pillars of the nonrepresentational theory that are taken into account for the staging.
The main objective to approach the space then, was to make an environment devoid of the originally intended meaning, a marketplace. As noted above, the only malleable element of this installation was the soundscape composition. It is therefore in the sound work that we could integrate intersemiotic interjections. Nevertheless, the sound material would be considered as elements of the composition but did not suggest a structure or mode present the work in a congruent way. The space, on the contrary, was always suggesting a structure. Even though the sunlight and the soundscape extended for twelve continuous hours each day, the visit itinerary for the artistic program was from 9am to 5pm. That frame suggests the necessity of composing a longer soundscape of at least five hours with a nonrepresentational structure as well. That is to say, the soundscape composition would last 300 minutes and be devoid of any structural division that could suggest a musical form or an easily graspable regular structure. On the contrary, the work would bring an embodied experience, free of listening suggestions of representation.
CONCLUSIONS
In Vitro is an installation that takes advantage of all connections that were already present in the Plaza San Mateo soundscape, the patrimonial building, and camera obscura before our intervention. It seeks to unify those three elements by making a work that could focus on the integral landscape of Plaza San Mateo in a multisensorial way, highlighting what is inaudible with a live projection, giving presence to what is invisible with a soundscape composition, and gathering all of that experience to reveal a place that is full of light, full of sound and full of public engagement.
Through this text, I have explored the connections that can be made between a philosophical proposal such as the rhizomatic thought (Deleuze and Guattari, 1972) and the process of an artistic production involving diverse materials (visual, audio, and spatial), which are interconnected through an intersemiotic interjection.
As seen in this small analysis of In Vitro, the creative process can be enhanced by interdisciplinary dialogues that incorporate the study of text and the study of philosophy. Such connections show us approaches to discuss the work of the creator not as a random and chaotic process, but as a systematized work that aligns subjective individual interpretations of artistic materials with those cultural and social interpretations that artists themselves make of such elements.
Finally, though this text presents just one case, it opens doors to review other artistic processes involving diverse artistic elements and proposing new approaches and methodologies to artistic analysis. Consequently, it can push us to investigate the possibilities that this integrative proposal for understanding and carrying out artistic production that unifies multisensorial creations.
REFERENCES
Aktulum, Kubilay. “What Is Intersemiotics? A Short Definition and Some Examples.” International Journal of Social Science and Humanity (2017)
Barthes, R. Roland Barthes. Paidós. (2004)
Deleuze, Gilles, and Félix Guattari. Capitalisme et schizophrénie. Paris: Éditions de Minuit, 1972.
García, E. Intertextualidad rizomática en la obra de Juan Carlos Tolosa. In Ortiz, J. (dir.), Cetta, P. and Guemba, G. (eds.) Cuadernos de Análisis y Debate sobre Músicas Latinoamericanas Contemporáneas Publicación temática anual, Año 3, Publicación III. Pp. 66-102 (2020).
Genette, Gerard., and Jane E. Lewin. Paratexts Thresholds of Interpretation. New York: Cambridge University Press, 1997.
Kristeva, Julia. [Sēmeiōtikē] Recherches pour une sémanalyse. Paris: Éditions du Seuil, 1969.
Secco, Leonardo. “La Musicalidad Del Paisaje Sonoro Urbano.” Dixit (Montevideo, Uruguay) no. 27 (2017): 4–. https://doi.org/10.22235/d.v0i27.1492.
Thrift, N. J. Non-Representational Theory : Space, Politics, Affect. London ;: Routledge, 2008.
Truax, Barry. Acoustic Communication. Norwood, N.J: Ablex Pub. Corp., 1984. (2nd ed). Ablex.
Williams N, , ‘Non-Representational Theory’, in International Encyclopedia of Human Geography (Second Edition), pp. 421 – 427(2019) http://dx.doi.org/10.1016/B978-0-08- 102295-5.10747-4
Lauro Zavala. “La Traducción Intersemiótica En El Cine de Ficción.” Ciencia Ergo Sum 16, no. 1 (2009): 47–54.
