Is Virtual Production Killing the Greenscreen?
Virtual production for live action filmmaking is well on its way to complete market proliferation, quicker than any filmmaking tool in recent memory. Even gimbals took a few years for mainstream adoption, whereas VP seems to have become an available and viable option nearly overnight. With COVID precautions and live events shutting down, using robocams and virtual sets made of live stage LED walls has become the new standard for more than a few studios around the world.
Those of us who have been watching the film world for the past decade know this isn’t truly an overnight success. Back in the early days, before we had a word for what it was, James Cameron used a bleeding edge proprietary system to visualize the entirely digital world of Avatar in real time, with real actor performances. Directors have been using Unreal and Unity to get a better idea of what the extended virtual set will look like in the final film for quite a while now, and The Jungle Book introduced us to the idea of a completely photoreal world built in a computer partially visualized on set. Up until about 2019, LED walls weren’t even in the cards as a realistic way to extend a scene. All those before relied on a technique now synonymous with visual effects: Greenscreen.
Whether a green or blue screen, keying and rotoscoping live footage to composite VFX in and around your scene is nothing new. It’s use alongside a live rendering engine, first in Avatar, and now in many films, live production TV shows, and news casts, is a modern extension of these techniques. Live keying coupled with a realtime engine is the precursor to the LED volumes we all drool over, but does this mean we kill the greenscreen altogether? Both techniques are now in use on a variety of sets across the world, and there are reasons for choosing one over the other, depending on a few key factors.
1. Price
While the benefits of a well-executed LED volume are apparent to any DP who has shot on one, the issue is how few cinematographers actually get on a set with that high of a budget. The price for just the panels needed for a small professional set extension is staggering, and those for a full volume are astronomical. The computers needed to drive each section of panels, the motion tracking system, the miles of cables, and the complex and often partially proprietary software cocktail to make it all work, all that needs to be in the budget. None of this includes the regular camera, lens, lighting, and grip packages needed for any production.
The specialized manpower and rightfully expensive contractors who can make this all work together correctly are an absolute necessity. Scrappy indie productions have made do with far less, but their process allows for errors and rethinking storyline based on limitations. Downtime for LED volumes is costly, and when you have millions of dollars invested in on-screen talent, you cannot afford to have anything go wrong.
Greenscreen is a tried and true solution for VFX, and has been for years now. While you still need a computer to run every simultaneous camera angle you want, there are no panels or extra computers to run them. These sets can be run incredibly lightweight with robotic camera operation, reducing the number of people needed on location to the single digits. I interviewed JP Connelly recently about his virtual production for Celebrity Show Off, and the results they were able to get with so few people on set are astounding.
Budgets for greenscreen virtual productions are not just lower, they can be a fraction of the price of LED volume productions. While the advantages of LED stages might outweigh greenscreen in other ways, the hit to your production budget is not one of them.
2. Complexity
If you read all of the Budget section, you already understand that LED virtual production is considerably more complex than greenscreen. Any engineer will tell you, the more independent systems have to be laid end-to-end, the higher the probability of failure. When you consider that some of these systems are custom made, the potential problems become clear.
Of course, an experienced team of VP professionals from a reputable consulting firm can mitigate this completely. There are so many ways to ruin an LED volume before it even gets built, it makes sense to take your ques from the experts. This does of course bring the budget back up, but it is better to spend the money building it now than trying to fix it later.
3. Post Processing
For the camera side of virtual production, the advantages of LED over greenscreen are the most apparent. Greenscreen has historically been difficult for directors and actors to adjust to, simply because there is nothing for them to interact with. Sir Ian McKellen had a notably terrible experience his first few days on The Hobbit because he found himself acting to nothing on a set where he was meant to be surrounded other actors. LED technology enables actors to see the previously hidden digital sets they occupy, and enabled directors to use the edges of their imagination as interactive set design.
What does this mean for visual effects? The first major change is the move from the previz scenes of greenscreen production to fully realized digital sets on an LED volume. Even for some greenscreen productions, the sets are ready to create final pixels on the day. This does mean the VFX department is a much more production-involved part of the crew, but it doesn’t mean there is no work in post. If you look closely on some behind the scenes shots from The Mandalorian, you will see it is not a digital or animatronic Mudhorn on the set, but a man in a grey motion capture suit acting as the digital character.
What this does mean is less work in changes after production is wrapped, and a massive help for the camera in terms of scene lighting, especially for shiny surfaces. C3PO had to be heavily edited and repainted in post for the Star Wars prequel trilogy due to his reflective body being used on greenscreen. For The Mandalorian, this was not an issue as the environment reflects naturally off the main characters’ armor and helm.
Final pixels in-camera will always be the main technical benefit of shooting on an LED stage or volume. The chance for the DP to actually see what the shot will look like is one I am sure many have wished for on modern sets. The opportunity to shoot on lenses with more character and still get the backgrounds you want is a pretty attractive prospect as well. While VP is enabling a similar process for greenscreen, having light from the digital set interact directly with your subject has a realism that is hard to replicate with other processes.
Is virtual production killing the greenscreen? To most pros, it sounds like an oxymoron; virtual production has been enabled by greenscreen for longer than VP has had a name. When it comes to VP, a greenscreen can get you around 60–70% of the visuals you want on set, and still get 95% of the final product you want after the fact. LED stages and volumes are still cost prohibitive for most filmmakers, and almost any stage has the option for greenscreen at reasonable production rates. For large scenes with a need for 4 story ceilings, Avengers Endgame style, there is no way an LED solution would suit your needs.
Will this always be the case? My best guess, greenscreen will be well out of fashion in about 20 years, with more streamlined processes for LED and much lower pricetags driving demand. It is still a mainstay of the VFX production world, but if LED production costs can eventually be reduced and film timelines accelerated, I see few benefits beyond post production flexibility to keep most pros using it. And hey, maybe I am wrong and flexibility is enough. I cannot count the amount of stories I have heard about post teams redoing finished shots for things like “seagull size”.
What do you think? Are greenscreens here to stay? Are LED volumes the future? Will AI enable perfect digital lighting with an array of lightfield cameras? I am excited to see where we go from here, and what innovations will enable indie filmmakers to take full advantage of this technology.
Thanks for reading, and as always, if you feel I have misrepresented information or flat out gotten something wrong, let me know! I want my work to be a resource for those learning about virtual production, and I know I can always improve. If you are the one learning, and you are wondering how to start, I have written an article on exactly how to do that.
