A new kind of special effects technology is currently revolutionizing the film world. At its heart are giant, high-resolution LED video walls that incorporate high-tech metals such as gallium and indium.
The Mandalorian, the first live-action series in the Star Wars universe, premiered on the streaming service Disney+ at the end of 2019. The space western about a lonely bounty hunter and his equally cute and powerful “foundling” triggered massive approval ratings. German newspaper Süddeutsche Zeitung was not the only one to call it “the best Star Wars story in a long time”. Most critics and fans felt reconciled with the franchise after the controversial last film trilogy.
Behind the scenes, the series also shimmers with superlatives. Unnoticed by most viewers, it set groundbreaking new standards in virtual film production. This is because StageCraft technology, also known as the volume, wasused for the very first time during filming. It was developed by Industrial Light & Magic (ILM), the special effects division of the Star Wars production company Lucasfilm. The most eye-catching part is an enormous LED video wall. Over six meters high and more than 50 meters wide, it surrounds the set in a three-quarter circle measuring 23 meters across. Virtual backgrounds are displayed on it and serve as a backdrop during filming.
At first glance, this is reminiscent of a trick technique from old Hollywood: rear projection. Here, actors play in front of a screen onto which separately shot film footage is projected. Car rides were often staged this way, as in Hitchcock’s film To Catch a Thief. But a closer look soon unmasked the two-dimensional montage – at the latest when the camera moved.
Intelligent Images Respond to the Camera
In contrast, the images in the volume seem almost intelligent. They aren’t static projections, but are generated in real time and interact with the camera. For example, if the camera pans to the left, the background on the LED wall adapts to its perspective correctly, as if it were filming a genuine environment. This is made possible by sensors that track the orientation and position of the camera.
The LED screen also serves as set lighting and provides the desired lighting mood and realistic reflections all at the same time. What’s more, every detail of the backdrop can be manipulated merely at the click of a mouse. Individual rocks, trees, buildings, and even complete landscapes can be moved; the position of the sun, time of day and weather can also be changed – including correct light and shadow effects.
Groundbreaking: The images on the LED walls create a perfect illusion. © & ™ Lucasfilm. The Mandalorian image courtesy of Industrial Light & Magic
It’s no coincidence that this is all reminiscent of a video game. That is because the basis of these images is the so-called Unreal Engine, a leading bit of software from the gaming industry, which can be used to create entire virtual worlds. Photorealistic, interactive 3D landscapes are assembled from millions of tiny triangles through which players can move their digital figures.
Still, most of the backdrops in The Mandalorian arenot completely computer animated. They are based on thousands of real shots taken before the actual filming. As the trade magazine American Cinematographer describes, they transferred images of real landscapes, objects, and textures to virtually created 3D objects to obtain the best possible result. So, the glistening desert world of Tatooine, Sorgan’s exotic forests and swamps, the lava flows on Nevarro – they’re all a clever computer montage. But the illusion is so perfect that the LED walls are indistinguishable from a real filming location for viewers of the series.
A Whole New Way of Filmmaking
For the development of StageCraft, ILM teamed up with Epic Games, the creators of Unreal Engine,and a number of other partners, including graphics card producer NVIDIA and Munich-based film technology company ARRI. One of the lead creative minds was The Mandalorian showrunner Jon Favreau. He had already experimented with individual elements of technology such as LED lighting and game engines inprevious films.
A video from ILM provides some insight into the filming of the second season of The Mandalorian andshows what the technology is capable of. The fact that practically every conceivable filming location can be brought into the studio and changed again at will opens up completely new possibilities, and not just on a creative level. Even if you have to invest in the complex technology first: Shooting in the volume simplifies logistics, saves time and money, and also reduces CO2 emissions. Series productions in particular, which have a smaller budget and a tighter schedule compared to films, can benefit from StageCraft. It’s not for nothing that The Mandalorian is often described as the first series to truly achieve cinema quality, for example in an article in the Washington Post.
StageCraft technologytakes virtual movie production to a new level and could largely replace the green screen. © & ™ Lucasfilm. The Mandalorian image courtesy of Industrial Light & Magic
Workflows became more efficient, Favreau explains in the video, with large parts of pre- and post-production coinciding with the actual shooting. As a result, 30 to 50 percent more script pages per day were shot, an “incredible improvement”. Expensive set construction is eliminated – as is time-consuming travel with the entire crew and tons of equipment. The location comes to the actors and not the other way around, Favreau told The Hollywood Reporter – an enormous advantage, especially during the pandemic, one which made it possible to continue shooting at all.
The End of the Green Screen Era?
Another cost saving factor: Compared to green screen, regarded for decades as the standard of digital film production, StageCraft significantly shortens the production process. With green screen technology – which ILM also pioneered, by the way – the actors play in front of a green (or more seldom blue) backdrop. This is later removed on the computer and replaced by another image. However, this makes the postproduction phase of the film expensive and tedious. Also, because the green color can reflect unappealingly on faces, costumes, and props. Staging The Mandalorian with green screen would hardly have been feasible simply because of the protagonist’s shiny armor. All of this is eliminated with StageCraft becausethebackground, lighting and reflections are already perfected during the shoot. Last but not least, the actors can rejoice: Their work is made easier because they can see the environment in which they are playing rather than acting in the middle of a green void.
Locations can be changed within a very short space of time, saving film teams time and money. © & ™ Lucasfilm. The Mandalorian image courtesy of Industrial Light & Magic
No Led Volume Without Indium and Gallium
ILM’s groundbreaking creation would not be possible without gallium and indium, as they are central components of all LED technology. Built into semiconductor compounds, these metals can be used to display almost any color and light intensity. The industry magazine Film & TV Kamera knows the technical details of the video wall that was used in The Mandalorian: Exactly 1,326 LED units are installed in it, supplemented by two movable ceiling modules with another 132 units. This results in many millions of individual light-emitting diodes (LEDs) with tiny amounts of high-tech metals that are indispensable for the function. The LED screens are ROE Black Pearl BP2 screens with a pixel pitch of just 2.84 millimeters. This is to minimize any possible moiré effects – those annoying streaks often seen in photos of screens.
The Future of Filmmaking?
Things didn’t stop with this first LED screen. ILM has optimized the technology and has since commissioned several volume studiosaround the globe. Productions shot there since then include the Star Wars series Boba Fett and Obi Wan-Kenobi, the Marvel blockbuster Thor: Love and Thunder, and The Batman from the DC universe. In addition, there are mobile StageCraft installations that can be set up anywhere; George Clooney used one for his Netflix film The Midnight Sky. Other production companies are following suit and building their own studios. One of Europe’s largest fixed LED installations has been located in Potsdam’s venerable Studio Babelsberg since last summer. The co-founder is Dark Ways GmbH, the production company of Baran bo Odar and Jantje Friese, the makers of the Netflix hit Dark. They filmed their new mystery series 1899 here.
LED screens are gaining ever more ground. Even though traditional techniques will probably always have their place – after all, even The Mandalorian also uses real backdrops and green screen technology – “The Volume” is becomingthe new standard. Trade media such as Indiewire and Film & TV Kamera speak of a game changer that will shape the future of filmmaking. LED screens – and with them tiny semiconductors made of gallium or indium – have set a far-reaching revolution in motion.
Images courtesy of Industrial Light & Magic
