Demolish Monsters on the Rocks: Prompting Through an Ensemble

bruce gilchrist

As interaction with corporate artificial intelligence increasingly becomes a precondition for contemporary life, artists need to see beyond Generative AI (GenAI) technology as a discrete tool that makes generic products. Instead, they can imagine combinatorial approaches and conceptual frameworks for AI-enabled artworks. Through my practice-based research, the act of prompting multimodal GenAI models has been informed by comprehending an assemblage as a “framework of instruction” held together through poetic alliances, within which the output from one component feeds the process of another. Practical experiments explored an interrelation of body, text, and predictive technology, where an algorithmic prediction of human action conjured “biometric poetry” that was used to stimulate a language model. Working with archival film footage and digital puppets animated with motion-capture files gave rise to the idea of a camera’s field of view – with its bounded contents acting like a key – eliciting value from a language model in a novel form of story making. Potential erroneous inferences were perceived as a new form of chance operation and a characteristic of algorithmic remix as defined by Steve F. Anderson. This method has been further developed in a project that combines performance, waste material, object recognition, and a language model to explore how the manipulation of garbage can be rationalised by a machine to produce poetic texts as a commentary to action portrayed on a screen.

18 Figures and Videos

Contents Back

Background Image (Contents)

Bruce Gilchrist, Idling-in-the-Unreal, 2021, still image from a video animation produced for the group exhibition “CAN WE EVER KNOW THE MEANING OF THESE OBJECTS”, Tickner Bell Young & Lebenson, Gallery 46, London E1, 2021. Copyright: Bruce Gilchrist.

Figure 1.

Stan Brakhage, Mothlight, 1963, YouTube, online video, 3:14, https://www.youtube.com/watch?v=UfBdTsljCXk&t=36s. Screenshot by Bruce Gilchrist. Copyright: The estate of Stan Brakhage.

Figure 2.

Schematic of project method, 2024. Copyright: Bruce Gilchrist.

Figure 3.

Screen recording of Python script and HAR analysis of British public information film clip, 2020. The source clip was extracted from the Ministry of Information film Coughs and Sneezes, directed by Richard Massingham, 1945. https://player.bfi.org.uk/free/film/watch-coughs-and-sneezes-1945-online. Open Government Licence v3.0. https://webarchive.nationalarchives.gov.uk/ukgwa/20230801101439/https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/. Copyright: Bruce Gilchrist.

Figure 4.

Textual output from a language model, superimposed on a film clip, 2020. The source film clip was extracted from the Ministry of Information film Coughs and Sneezes, directed by Richard Massingham, 1945. https://player.bfi.org.uk/free/film/watch-coughs-and-sneezes-1945-online. Open Government Licence v3.0. https://webarchive.nationalarchives.gov.uk/ukgwa/20230801101439/https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/. One of the British public information films that was used to stimulate the HAR algorithm to make Synset_Gloss, 2020. Copyright: Bruce Gilchrist.

Figure 5.

Nye Thompson, CKRBT, 2019. Installation photo, Watermans Gallery, London. https://nyethompson.net/works/the-seeker-ckrbt.html. Copyright: Thompson, Nye. Courtesy Nye Thompson.

Figure 6.

Nye Thompson, CKRBT, 2019. Photograph taken by Bruce Gilchrist of the CKRBT process schematic, Watermans Gallery, London.
Copyright: Thompson, Nye.

Figure 7.

Singh, Amarjot, Devendra Patil, and S. N. Omkar. 2018. Eye in the Sky: Real-Time Drone Surveillance System (DSS) for Violent Individuals Identification Using ScatterNet Hybrid Deep Learning Network, https://arxiv.org/abs/1806.00746, 2, fig. 1. Copyright: Singh, Amarjot, Devendra Patil, and S. N. Omkar.

Figure 8.

Amarjot Singh, Eye in the Sky: Real-time Drone Surveillance System (DSS) for Violent Individuals Identification, 2018, YouTube, online video, 2:00, https://www.youtube.com/watch?v=zYypJPJipYc. Screenshot by Bruce Gilchrist. Copyright: Singh, Amarjot.

Figure 9.

HAR Misclassification – driving tractor, 2020. A composite of video stills from the Kinetics 400 dataset “driving tractor” category and the British public information film The Spirit of Dark and Lonely Water, directed by Jeff Grant, 1973. https://player.bfi.org.uk/free/film/watch-lonely-water-1973-online. Open Government Licence v3.0. https://webarchive.nationalarchives.gov.uk/ukgwa/20230801101439/https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/. Copyright: Bruce Gilchrist.

Figure 10.

HAR Misclassification – swimming backstroke, 2020. A composite of video stills from the Kinetics 400 dataset “swimming backstroke” category and British public information film The Spirit of Dark and Lonely Water, directed by Jeff Grant, 1973. https://player.bfi.org.uk/free/film/watch-lonely-water-1973-online. Open Government Licence v3.0. https://webarchive.nationalarchives.gov.uk/ukgwa/20230801101439/https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/. Copyright: Bruce Gilchrist.

Figure 11.

HAR Misclassification – abseiling, 2020. A composite of video stills from the Kinetics 400 dataset “abseiling” category and British public information film A Boy Goes Cycling, an amateur film and Public Information Filler, 1963. https://player.bfi.org.uk/free/film/watch-a-boy-goes-cycling-1963-online. Open Government Licence v3.0. https://webarchive.nationalarchives.gov.uk/ukgwa/20230801101439/https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/. Copyright: Bruce Gilchrist.

Figure 12.

Oeuvre, 2021. A 17-volume corpus of Michel Foucault visualised using Word2Vec Word Embeddings and t-SNE. Copyright: Bruce Gilchrist.

Figure 13.

Schematic showing Synset_Gloss workflow, 2021. Copyright: Bruce Gilchrist.

Figure 14.

Schematic showing No_m_oN workflow, 2021. Copyright: Bruce Gilchrist.

Figure 15.

Schematic showing Idling-in-the-Unreal workflow, 2021. Copyright: Bruce Gilchrist.

Figure 16.

Three different fields of view (FOV) with tabulated textual output from the assemblage, 2021. Copyright: Bruce Gilchrist.

Figure 16/FOV–1.

Three key pieces of information for FOV-1 have been tabulated: first, the text label produced by the HAR system; second, text outputs from a Markov chain generator using the same HAR label; and third, a selection of autocompleted outputs from the language model. Copyright: Bruce Gilchrist.

Figure 16/FOV–2.

Three key pieces of information for FOV-2 have been tabulated: first, the text label produced by the HAR system; second, text outputs from a Markov chain generator using the same HAR label; and third, a selection of autocompleted outputs from the language model. Copyright: Bruce Gilchrist.

Figure 16/FOV–3.

Three key pieces of information for FOV-3 have been tabulated: first, the text label produced by the HAR system; second, text outputs from a Markov chain generator using the same HAR label; and third, a selection of autocompleted outputs from the language model. Copyright: Bruce Gilchrist.

Figure 17.

Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi, Plastic Prāyaścitta, January 2023, film production photo by Rahul Naag. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Figure 18.

Poster for Poetics of Garbage created by the BeFantastic Beyond Commission using an image created by Bruce Gilchrist, 2022. https://befantastic.in/befantasticbeyond/. Copyright: BeFantastic Beyond.

Figure 19.

Schematic showing Poetics of Garbage workflow, 2024. Copyright: Bruce Gilchrist.

Figure 20.

The Making of Plastic Prāyaścitta, January 2023, film production photo by Purva Joshi. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Figure 21.

Poetics of Garbage, 2023, installation at Bangalore International Centre as part of FutureFantastic festival, photo by Bruce Gilchrist. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Figure 22.

Figure 23.

Poetics of Garbage, 2023, installation at Bangalore International Centre as part of FutureFantastic festival. The YOLOv7 label “potted plant” seeds a line of text generated by the RNN text generator, photo by Bruce Gilchrist. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Figure 24.

Poetics of Garbage, 2023, installation at Bangalore International Centre as part of FutureFantastic festival. The YOLOv7 label “toilet” seeds a line of text generated by the RNN text generator, photo by Bruce Gilchrist. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Video 1.

12. bowling, 2020, 02:02. Development video for Synset_Gloss. No sound. The video dataset was originally downloaded from the Department of Electrical Engineering, University of Texas at Dallas: https://personal.utdallas.edu/~kehtar/UTD-MHAD.html. Copyright: Bruce Gilchrist.

Video 2.

Synset_Gloss, 2020, 07:48. Video and animation with stereo sound. Copyright: Bruce Gilchrist.

Video 3.

Video 4.

Idling-in-the-Unreal, extract, 2020, 02:59. Animation with stereo sound. Copyright: Bruce Gilchrist.

Video 5.

Idling-in-the-Unreal, extract, 2020, 06:05. Animation with stereo sound. Copyright: Bruce Gilchrist.

Video 6.

Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi, Plastic Prāyaścitta, extract, 2023, 01:11. Video and animation with stereo sound. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Video 7.

Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi, Plastic Prāyaścitta, extract, 2023, 01:57. Video and animation with stereo sound. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Video 8.

Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi, Plastic Prāyaścitta, extract, 2023, 01:22. Video and animation with stereo sound. Copyright: Arora, Aashna, Bruce Gilchrist, Chaitali Kulkarni, and Thaniya K. Mahalakshmi.

Tip

Tip