Tandem: Imagining Art in Collaboration with an Intelligent Machine
Expression through art is core to our very being. From cave paintings to digital media, we’ve explored various ways of expressing through art. Can something as fundamentally human, be carried out along with an intelligent machine? How can machines be intimately involved in the process, yet not hijack our own intentions? What would happen if machines were to take part in the imagination process of creation, offering their own creative interpretations based on a visual experience similar to humans? Can this lead to a step beyond computer augmentation as we know it, where computers become truly involved in the subjectivity of the process itself? How can AI transcend from an intelligence capable of solving complicated tasks, to rather being an Artificial Imagination (A.I) of sorts, leading us to new, serendipitous discoveries. How would we perceive our role in such a medium? These are the thoughts and questions Tandem tries to explore.
Tandem is a software system where a person’s drawing input is ‘imagined’ upon by a computer to suggest an outcome, thereby interweaving the two in a creative dialogue. The human artist can also exercise control over personalities of Tandem, from joyous to furious, or sad to dreamy, changing its output accordingly. By making creative expression a collaborative process between humans and intelligent (imaginative) machines, Tandem aims to provide a means of exploring the realms of machine intelligence and imagination in a way very natural to us, that of drawing.
Workflow
Tandem is a drawing application. As a person creates a drawing, they can choose different personalities of Tandem and let Tandem imagine on top of their drawing. Tandem produces an output driven by the human drawing and completes its own interpretation of it. A person can see the output coming along as Tandem goes through its iterations in producing it. The human can edit the output, or go back and edit their own input and continue the creative dialogue.
This project began with Arnav Kapur as a result of a lot of conversations with him around AI and creativity!
Here’s a screenshot of the Tandem drawing application with the output:
Tandem became part of the world’s largest computer science museum– the Heinz Nixdorf Museum’s (HNF) exhibition on AI in 2018. HNF is in Paderborn, Germany.
Tandem at HNF Museum
Some input-output pairs of Tandem:
Tandem v1 1. 35×18 inches. Archival Print on Paper. 2016. (L- Artificial Intelligence Output)
Tandem v1 1. 5×2.57 inches. Archival Print on Paper. 2016. (R- Human Input)
Tandem v2 1. 35×19.5 inches. Archival Print on Paper. 2017. (L- Artificial Intelligence Output)
Tandem v2 1. 5×2.7 inches. Archival Print on Paper. 2017. (R- Human Input- Blank)
Tandem v2 2. 35×19.5 inches. Archival Print on Paper. 2017. (L- Artificial Intelligence Output)
Tandem v2 2. 5×2.7 inches. Archival Print on Paper. 2017. (R- Human Input)
Tandem v2 3. 35×17.5 inches. Archival Print on Paper. 2017. (L- Artificial Intelligence Output)
Tandem v2 3. 5×2.5 inches. Archival Print on Paper. 2017. (R- Human Input)
Tandem v2 4. 35×19.5 inches. Archival Print on Paper. 2017. (L- Artificial Intelligence Output)
Tandem v2 4. 5×2.7 inches. Archival Print on Paper. 2016. (R- Human Input- Blank)
Here’s some more outputs (human drawing on left, Tandem output on right):
The first is with a ‘dreamy’ personality, the second with a ‘happy’ personality.
Here’s a video of Tandem in action (the first instance is with a ‘happy’ personality, the second with a ‘dark’ personality):
Different versions of Tandem have been exhibited at the Otherly Space/Knowledge exhibition at Asia Culture Center (curated by Kazunao Abe-san), Art Center Nabi (Seoul), Alt-AI (New York), BeFantastic Festival (India), Why the Future Still Needs Us at QUT Art Museum (Australia).
Here’s some images from the Otherly Space/ Knowledge exhibition.
Here’s some from Art Center Nabi.
Here’s from QUT Art Museum.
Following are some more representations/ experiments with Tandem:
This one is about a different mode- a pencil sketch mode instead of using colors:
This is an attempt at combining Tandem with an actual painting setup in the same physical space:
Here’s some experiments with the computer also producing a caption and some text statement about the drawing:
Another feature of Tandem is the possibility to present historical paintings made by artists that closely resemble the one made by you and Tandem. This can be a nice way of discovering paintings, giving some us context of the art we produce (and making us feel good about our art output!):
Did some experiments with other visualizations of Tandem, using HED (edge-detection algorithm).
Some other A.I Art Experiments aligned with Tandem:
I have also been carrying out some other experiments with combining drawing and AI. Following are some early experiments. One particular area of interest has been to combine it with ‘cultural objects’. Can we teach machines about different cultures, and as a result make that a medium for us to discover different cultures. Can this lead to ‘exploration through creation’, where a computer presents us with different cultural designs and patterns as we sketch different objects. The larger goal of this is also to question the cultural background of AI, based on what datasets we use to train it.
Following is early beginnings, where i input various Japanese and African vases to a computer. When a person sketches a vase shape, the computer presents them 2 outputs- with Japanese kind of patterns and African ones. (NB- this is in no way an extensive study of cultural patterns and their meanings).
Something similar with African masks:
I have also been interested in how style transfer techniques can be used in terms of creating cultural hybrids/ culture transfers, possibly speaking to the coming together of different cultures and blurring of lines between them, in today’s age of travel and technology.
Here are some initial experiments with adding patterns from different cultures to the Dakishi T-shirt from Africa.
Here’s with an aboriginal pattern:
Some more:
Here’s one with a mask!