Synthetic media rely on machine learning tools that perform a range of tasks. They enable micro-targeted systems of distribution; streamline production and post-production workflows; and animate, edit, or even create entire audiovisual works from a text prompt. Many of the hazards of synthetic media, such as malicious deepfakes, have been well documented and need further attention. Still, AI elicits a special kind of anxiety for the film and TV industry’s creative classes. The question now on everyone’s mind is whether feature-length films made by text-to-video generators will eliminate the skilled labor of screenwriters, graphic artists, editors, and directors. It is doubtful that Hollywood studios will launch a major lineup of AI-generated features anytime soon. Nor will viewers likely want to completely forego the experience of collectively shared films or series in favor of bespoke entertainment they create with a few sentences of prompt engineering. Even as text-to-video software continues to improve at an extraordinary rate, it will never replace the social elements crucial to the product Hollywood makes and the culture that surrounds both gaudy blockbusters and gritty dramas alike. To produce a film is to give shape to the multiple human narratives involved in its fashioning. People—the deals they make, the complex biographies they bring to the script, their chemistry or friction on set, the aesthetic influences that inform their approach to sound design or cinematography—make a picture what it is. (A more pressing concern is that studios will use algorithm-driven predictive analytics to greenlight only those projects they believe are sure to make money, leading to less diversity of form, story, and talent.) At the same time, filmmakers are gravitating to AI to expand their craft and create new forms of moving image art. AI aids in the speed and precision of workflows in ways that are important, but not always visible to spectators. Machine learning tools can organize footage so editors can more easily recall particular camera angles and scenes of dialog. And on the other side of a project’s life, restoration efforts involving algorithms remove dirt and scratches from old prints and repair the ravages of warp and flicker. One of the most prominent ways AI has surfaced on screen has been through VFX work, to shape complex performances. The Wētā FX software Massive has helped effects artists capture the seemingly “unfilmable,” especially on the macroscale. Beginning with the creation of digital hordes of orcs and humans for the realistic combat in The Lord of the Rings: The Two Towers’ (2002) Battle of Helm’s Deep, Massive has since been responsible for expansive collections of lifelike entities, from the shiver of sharks in The Meg (2018) to the swarms of flying demons in Shang-Chi and the Legend of the Ten Rings (2021). A remarkable if ghoulish extension of this work can be seen in “synthetic resurrection,” where a deceased actor or historical figure is brought back to life to play a role in the present. It’s likely that we’ll see deceased figures come back around for the purpose of an eye-catching cameo, a supporting part, or to smooth over the narrative complexities of a multi-year series or franchise. It would be surprising if AI-as-lead became the norm, however. Even if it becomes easier to negotiate “likeness rights,” these simulations will still plunge us into new terrain within the uncanny valley. Viewers’ discomfort would not necessarily stem from the image itself, but from the cognitive dissonance of trying to square the knowledge of the character’s digital construction with how real they seem on screen. Synthetic resurrection has been particularly contentious in the realm of documentary. While the field positions itself against media deception and manipulation, filmmakers have begun to use AI in subtle ways. One example is the vocal synthesis employed in The Andy Warhol Diaries (2022), an otherwise formally conventional nonfiction portrait of the artist. Director Andrew Rossi, in collaboration with Resemble AI, has Warhol narrating his ruminations to assistant and friend Pat Hackett via an intermittent voice-over. To avoid the controversy surrounding Morgan Neville’s simulation of Anthony Bourdain’s voice for his 2021 biodoc, Roadrunner, Rossi secured permission from the Warhol Foundation and alerted viewers to the use of the technology in a disclaimer. The result is a redoubled emphasis on traditionally “Warholian” preoccupations, encouraging viewers to reflect on the nature of mediation, the fabrication of celebrity, and what can be simultaneously revealed and concealed by public forms of address. AI has also become increasingly important in human rights documentaries. For Welcome to Chechnya (2020), a film about the oppression of LGBTQ activists there, VFX supervisor Ryan Laney drew on deep learning tools to craft digital faces for the film’s subjects. Laney used volunteer activists in New York to create composite faces that protected the subjects’ identities while still allowing viewers to emotionally connect with them on screen. Laney expanded on these techniques in Faceless (2022), Jennifer Ngo’s vérité-style documentary about young activists on the front lines of the 2019 pro-democracy protests in Hong Kong. Here, Laney scrambled the relationship of measurements between specific points on the protagonists’ faces to stymie attempts to identify them using biometric scans. The proliferation of “generative” text-to-image and text-to-video software is pushing AI into a new phase. Instead of looking at the visuals from models like DALL-E 2, Midjourney, and Stable Diffusion as fixed and isolated objects, we could consider how these tools might operate as part of a larger creative process. Synthetic imagery is sure to become central to preproduction. Screenwriters will be able to use AI-generated imagery for their pitch decks to evocatively establish the mood and feel of a project and position it within a larger genre. Likewise, concept artists will benefit from the back-and-forth tweaking of prompts and visual outputs as they flesh out a film’s narrative arc in the early stages of storyboarding. Generative AI might also expand the “previs” process of transforming flat images of material environments and character interaction into 3D approximations of scenes. Generative image software also offers artists new ways to connect with audiences. For example, German technologist Fabian Stelzer engaged with Midjourney, Stable Diffusion, and DALL-E 2 to create a core set of images for a sci-fi fantasy about the discovery of a malicious salt substance in outer space. He also used GPT-3 to assist with the script and voice-generators Murf and Synthesia for the narration. Stelzer then published short sequences of this film, SALT, on Twitter (what he calls “story seeds”), along with a call for followers to vote on further developments in the unfolding narrative. The result is a crowdsourced essay film that takes some loose formal and thematic inspiration from Chris Marker’s landmark experimental essay film, La Jetée (1962). Generative AI also offers a way to blend graphics and performance, breathing life into genres such as the music video. Belfast-based computational artist Glenn Marshall won the Jury Award at the Cannes Short Film Festival for his AI-driven film, The Crow (2022). Marshall started with Duncan McDowall’s captivating video of Dorotea Saykaly dancing in a black shawl in a decaying industrial building. He then fed frames of the video to OpenAI’s CLIP, a neural network model. The result is a work of clever style transfer, where every live-action frame is given a painterly treatment that transforms the dancer into a crow-like figure, encouraging viewers to reflect on the relationship between human and nonhuman animals, technological process and aesthetic experience. Part of what is exciting about these projects is their strangeness and messiness, the ways they flaunt their own fabrication. These projects point to the productive frictions of mixed-media and cross-platform practices. Much of the utopian rhetoric around generative AI has to do what it means for the individual maker. Never before has an amateur or seasoned professional been able to build such an elaborate project on such a small budget in such a short amount of time. This line of thinking resonates with the long history of mythmaking within Silicon Valley and Hollywood, two places that love and love to loathe their geniuses. The more promising dimensions of this software and the debate surrounding its use, however, have less to do with visions of reinvigorated auteurism than the fact that it requires a constellation of skilled labor, not only to make it, but also to deploy it in innovative fashion. We might see a broader reframing of authorship as VFX supervisors—not to mention the computer scientists, concept artists, engineers, and animators they work with—become increasingly responsible for the movements and expressions of the characters on screen, as well as the look and feel of the world they inhabit. It remains to be seen whether this is an inherent feature of this new media or a first stage, much like the early stages of cinema itself.
