Demolish Monsters on the Rocks: Prompting Through an Ensemble

bruce gilchrist

As interaction with corporate artificial intelligence increasingly becomes a precondition for contemporary life, artists need to see beyond Generative AI (GenAI) technology as a discrete tool that makes generic products. Instead, they can imagine combinatorial approaches and conceptual frameworks for AI-enabled artworks. Through my practice-based research, the act of prompting multimodal GenAI models has been informed by comprehending an assemblage as a “framework of instruction” held together through poetic alliances, within which the output from one component feeds the process of another. Practical experiments explored an interrelation of body, text, and predictive technology, where an algorithmic prediction of human action conjured “biometric poetry” that was used to stimulate a language model. Working with archival film footage and digital puppets animated with motion-capture files gave rise to the idea of a camera’s field of view – with its bounded contents acting like a key – eliciting value from a language model in a novel form of story making. Potential erroneous inferences were perceived as a new form of chance operation and a characteristic of algorithmic remix as defined by Steve F. Anderson. This method has been further developed in a project that combines performance, waste material, object recognition, and a language model to explore how the manipulation of garbage can be rationalised by a machine to produce poetic texts as a commentary to action portrayed on a screen.

13 Fluid Narratives

The digital puppet’s gentle, idle motion in Idling-in-the-Unreal consistently elicited sea-related text from the ensemble, creating a sense of bodily response to the movement of water and implying an interplay between visual and textual elements.

Contents Back | Next

Back | Next

In the beginning section of this clip (Video 5), the gentle rocking motion of the digital puppet, produced by its idle animation, consistently triggered the generation of sea-related text sequences, giving the impression of a body responding to the motion of water. This appears to be underscored by a complex interaction between visual elements and textual cues within the ensemble. By tracing the productions of the workflow, specifically to the small corpus created by linking the HAR system labels to WordNet, phrases were found with semantic ties to liquids and water, such as “An arm of the sea”, “A branch of the sewer”, “Stretch the soup by adding more cream”, and “Adulterate liquor”. These liquid-themed phrases were used to stimulate the Markov chain model, the outputs of which were used to prompt the language model to generate vivid, abstract, oceanic narratives. The Markov chain outputs serving as prompts for the language model are highlighted in bold:

Unfold the sea, and crate it underwater to form a dynamical thing [...]
Unfold the sea, that can and should carry the rum and the whale for the expenditure of the money of the workers in the workshops and in the manufactories [...]
Stretch the sea at will, demolish monsters on the rocks, crush mines underfoot, kill envious rivals, scour the seas with rain and storms that never cease [...]
Stretch the sea by setting up a network of short, continuous sails [...]
Increase in the sea’s buoyancy, which adds to the general circulation, in due time all the surface of the water is caught up in a dense mass, which is termed a “cascadian” [...]

Video 5. Idling-in-the-Unreal, extract, 2020, 06:05. Animation with stereo sound.

Tip

Tip