Virtual Production
The film locations industry is going to go through a big change. Location scouts will be sent to find the perfect location, just as they are now, but the past concerns about crew parking, size of the rooms for camera and lighting, availability of hotels for cast and crew etc., plus the proximity to other locations, will all go out of the window. Instead, photos will be taken and a director will approve the film location photos. Once the location has been approved, the locations team will go back to the location with their new photogrammetry tech team and spend a day or two capturing megascans of the location and all the items in the room. No big crew. No massive transportation team. That location, forevermore, captured on a hard drive.
Disney uses Epic Games’ Unreal Engine to render real-time sets in The Mandalorian
The Mandalorian, a Disney+ exclusive, is the first-ever live-action Star Wars television series in the history of the franchise. It is also the first production of the property that uses Epic Games’ Unreal Engine. The worlds of filmmaking and video game technology are colliding. The Mandalorian set had LED video walls used to provide “in-camera composites.” The SFX team projected pre-rendered content like environments on the LED walls, which served as dynamic green screens of sorts.
The Mandalorian
One of the most impressive parts of The Mandalorian (besides Baby Yoda, the heartwarming story and the second season finale’s surprise) was the production design, with realistic and varied locations, sets, and effects despite the comparatively lower budget for television production as opposed to a blockbuster movie production.
More than half of the first season of The Mandalorian was shot using Industrial Light & Magic’s (ILM) new StageCraft technology, which uses giant 20-foot-tall LED video screens to create fully digital environments as in-camera sets and backdrops. ILM worked with Epic Games’ Unreal Engine to build these purely digital environments which were then projected around the actors and physical set elements (like spaceship parts or speeder bikes) to create the finished project. As part of that effect, those LED screens are actually then shot on camera for a seamless effect that replicates location shooting without the costs. (The 3D environments are specifically lit and rendered from the perspective of the camera to accomplish that.)
There are also real benefits to the virtual sets compared to a traditional green screen setup. Since the actors are actually on the set with the displays, they can see and react to things in the background while they’re actually shooting, instead of imagining effects that will be added later.
It also means that the lighting from those digital sets is present on set and interacting with the physical elements, like the actors and props. So shots where Mando and Baby Yoda sit around a fire, with the sunset shining off Mando’s armor still look right because that light is actually there on set. It’s just coming from an LED screen instead of an actual sun.
Additionally, because the “sets” are just digitally projected 3D environments, they can be moved around and edited on the fly. Don’t like where a mountain is in the background for a particular shot? Simply move it around, or just delete it entirely.
ILM had already been using effects like this on existing films, like Star Wars’ Solo, where the “windows” of the Millennium Falcon were digital screens, allowing for the lighting effect of the hyperspace jumps to be projected on the actors’ faces in camera. But The Mandalorian took the technology to a much bigger scale, with a 270-degree semicircular LED video wall and ceiling that created a 75-foot-wide set.
The Mandalorian’s first season is just the start for this new technology. ILM is making StageCraft available for use by filmmakers and showrunners worldwide as an end-to-end solution, meaning that the digital set technology could be showing up in a lot more TV shows and movies in the future.
In the video, The Mandalorian production company shows the power of its virtual sets. It can quickly make adjustments to the virtual backgrounds and then capture the results with the camera. This means that ILM no longer has to composite together separate effects and green-screen footage. Instead, it can point the lens at the action inside of one of its “volume” LED production sets. Check out ILM’s video to get the full look at StageCraft.
Innovative Technology
Jon Favreau turned to video game technology in the filming of The Mandalorian. The Disney+ series uses what’s constantly been referred to as “groundbreaking new technology” for the film industry, Something ILM calls “StageCraft.” Using Fortnite creator Epic Games’ Unreal Engine, the filmmakers render a scene in realtime and then project it onto massive LED screens that encompass the acting space. These scenes are then tracked with pixel-perfect accuracy to the camera, keeping perspective and parallax just right no matter what movements are made.
There are a ton of advantages to this method of filming locations and backplates for scenes. Instead of needing to scout locations or build complex sets, filmmakers can just create the perfect location themselves and have them ready for filming within 24 hours. If adjustments need to be made, such as the positioning of certain elements, coloration, or lighting, they can make those changes on the fly. Time of day also isn’t an issue. The production team described the ability to shoot a 10-hour day at dawn because they can simply set the scene when they want it. No more waiting for perfect weather or lighting.
For the actors, it’s a much more immersive way of filming. The Mandalorian’s creators have talked about almost forgetting that they were on a set when in the middle of a scene. And instead of needing green screens to make post-production edits to the environment, everything in the scene is already captured at the time of filming. This also means that lighting, reflections, and color are more accurate, requiring less post-production and processing to fix those issues, and giving the creators more freedom of choice to make creative effects decisions while filming rather than in the editing process.
Using Unreal Engine and LED screens is a perfect solution for faster TV show production, allowing shows like The Mandalorian to have big-budget sets and locations and get everything filmed and edited faster. The production team continued to expand on the idea throughout The Mandalorian’s first season, even bringing physical set pieces right into the LED screen stage. This added elements of dynamic lighting and reflections off of the physical objects, more physical interaction to the scenes, and allowed them to further blur the lines between what was digitally created and what was actually there on the set.
Images: Industrial Light & Magic and Lucasfilm Ltd.
MicroLED Displays
Sony has announced a pair of new Crystal LED modular direct view displays that could make virtual movie production – of the kind used to create the digital sets in The Mandalorian – a far more widespread method of filmmaking. The displays, developed in collaboration with Sony Pictures Entertainment, comprise several modular panels that use Micro LEDs to produce enormous digital images, allowing actors to exist in virtual movie worlds without the need for traditional green screens. The technology means directors can control set surroundings without moving between locations or constructing expensive building projects, which typically eat into large portions of a movie’s production budget. Sony announced that its B-series displays are geared towards professional applications – including virtual set production – given their anti-reflective coating and high degree of brightness. Sony says the displays can operate at an average of 1,800 nits, which is in the upper range of brightness even for Samsung’s ultra-bright QLED displays. For comparison, even Apple’s Pro Display XDR (referring to its “extended dynamic range”) reaches a peak of 1,600 nits.
Sony’s new displays will allow greater visual fidelity to actors and filmmakers tasked with creating authentic, convincing worlds. It’s the reason why The Mandalorian sets seem so real – the actors are able to deliver their lines while the sands of Tatooine whirl around them, rather than working alongside giant green panels. If actors are convinced of their own existence in these virtual worlds, then so will viewers. Displays like these allow for more natural light emission, doing away with the false lighting required from traditional filmmaking methods.
Reflections, too, are more realistic. Typically, post-production teams are forced to spend hours editing out pesky reflections caused by green screens, inserting false effects to create the illusion of natural light. With Sony’s virtual displays, reflections seen in movies will actually be those caused by the light of the actors’ surroundings, creating a more authentic, immersive visual experience.
Images: Sony Electronics
For decades, film and TV productions have used green and blue screens to place actors into new environments. Now, LED walls are revolutionizing this process by projecting 3D environments in real time behind actors to provide the illusion of being in a physical location. These methods were put to the test on Disney’s “The Mandalorian,” of which over half was filmed indoors on a virtual set. This process of combining traditional cinematography techniques with advanced world-building technology effectively eliminates the need for a green screen.
MX MOVIES
More Demand for Sound Stages
Movie studios will be even higher in demand than now as the future of film productions (both movies and TV series) will see a huge number of projects vying for the limited amount of sound stages that are available around the world, and especially in Canada because of British Columbia’s tax credits and beautiful exteriors. With sound stages, the crew will not have to travel and the environment can be controlled, thus improving economies of scales achieved through reducing unit moves, etc.
The use of sound stages is nothing new, but what is revolutionary is the use of 3D photo-real worlds. Green screen and LED screen technology such as Lux Machina’s high-resolution 8K video backdrops will become common filming practice in the industry rather than film crews camping on multiple locations with large basecamps and having to deal with the limitations of a real world environment.
For major studios, they will retain the worldwide rights for that location and they will be able to build their own location libraries with locations being able to be reskinned and adjusted based on the production designer’s and director’s vision. Major famous locations and environments will do exclusive deals with libraries for their mega scanned 3D environments.
MOVIE STUDIO DEVELOPMENT
REQUEST INFORMATION
Our team of professionals in the film industry is happy to assist you with your questions and work as your consultant in developing your movie studio. We will request a Confidentiality Agreement and an Engagement Agreement to be signed before further discussion due to the confidential nature of the information shared and the detailed navigation of your process in building a purpose-built movie studio, warehouse conversion, or FX and broadcast stages.