You know the green screen (and blue screen). You know it becomes the:
Mist-covered planet - Deserted jungle
- Background for a bar - Packed athletic stadium
- Panoramic waterfall What happens when the imaginary planet...is
already there? “The moment you step in the middle of the
Volume, you're just, you're just there.” “The Volume” is the epic sounding name
for the combination of high resolution LED panels — imagine awesome huge TV screens
wrapping around a stage — the physical set design matched to the panels, and 3D models
plopped into an environment the same way they do in a video game.
Then it can respond to camera movement to simulate the real world.
Disney +’s the Mandalorian, a live action Star Wars TV show, used this technique.
“My name is Charmaine Chan and I'm a lead compositor at Industrial Light and Magic.
Is it, is it physically, like, confusing being on this set?
“Oh absolutely, I mean, the thing is like you're, you're shooting all day, let's say
in the same exact scene and like you're at that location, it doesn't feel like it's something
fake. It just feels like the extension of a regular stage. You gotta be careful because
there are times people don’t see where the edge of the stage is and where the LEDs are.”
You have Wiley coyote and Roadrunner situations where somebody is like running into the wall
or something. “Yeah. We, we definitely made sure that
no one's running in that stage because of that reason.”
Charmaine is credited as part of the Brain Bar - the group of visual effects artists
that operated this system. One might adjust models, like a rock or spaceship,
in the panels, while another might tweak live animations, like a burning fire. Charmaine
often adjusted color. “It was funny cause it, it looked very much
like, you know, back in the day when you would have
telephone fundraising stuff, like on PBS and it was just like rows of people phone's ready
to go, but instead of phones we had computers and our walkies.”
As Mandalorian VFX supervisor Ian Milham Tweeted, the set crew and Brain Bar operating the panels
let them radically change environments in just a few hours or be on set...as they launched
it into hyperspace. “My normal working life is very much behind
the computer in a dark room, somewhere in the corner. Now I'm actually in there with
the gaffer, with the prop designers, with the set designers, most people that we would
never see because we’re in the post production process. It was very exhilarating.”
But sets like this one weren’t just fun for Charmaine. They helped remove creative
roadblocks. As a compositor, we're the ones who kind of
take all the renders, take all the CG elements and put them together to make it look like
it's a seamless, integrated photo. So think of it as like advanced Photoshop, But we're
dealing with moving imagery.” Charmaine worked on this scene in The Last
Jedi. “We get this footage of Kylo in front of
a green screen. If you're lucky, this green screen will be evenly lit, with no seams.
And it's piece of cake. That's never the scenario. We’re spending the time, almost frame by
frame, making sure we can remove that green screen so that we can put Kylo on top of that.”
Removing a green screen is actually still pretty hard. For one, it doesn’t work with
green characters. “Yoda’s green.”
Removing one solid color — or “keying” can look good, but you still need detail work.
See how these fine branches just disappear. The perspective of the background also doesn’t
naturally change - that has to be designed into the final composite.
Ditching the greenscreen and projecting or playing the image behind the actors…. gets
you closer, but not quite there. No!
You can get detail and an illusion of depth and better light. Instead of green screen
spilling on the actor, you get blue sky, and red desert actually lighting them.
That basic technique has worked in everything from 2001 to Oblivion, but you miss the proper
perspective shift, or parallax, in the background, since it’s just a video playing on a screen.
The volume tackles some of those problems. You can also adjust light and objects on the
fly. And the reflections actually work, because
they are reflecting the other screens instead of green screen - which was especially important
for Mando — the show’s main character. “His whole armor was reflective from head
to toe, whether it be his pauldron or his helmet, it was just like, you can't avoid
seeing things being reflected. So creating this volume where we literally could close
up the whole thing into one giant circle and have an environment all across these screens.
We were getting exactly what we wanted to out of his helmet.”
The brain bar could focus on details that made the final product as seamless as possible
— which was still a lot of work. “I would go in and whether it be a rock
or a barrel or something, I would try to color correct it to match what was on the set. But
where color correction was more important was when we're dealing with the bigger parts
of the set. So whether that be the dirt on the ground versus dirt in our digital scene.
And the lighting from the scene affected the dirt on the ground. And we would have to like,
because we had a blue sky and suddenly now there's all this blue on this rock. We would
have to color correct the ground and the rock to also have just as much blue as the blue
that we just introduced. Before they started shooting, I would have. Five to 10 minutes
to have that all lined up and ready to go.” I can imagine that, that there are some creative
breakthroughs that this makes possible for, for your job. I'm wondering what would those
be with this technology? “I’'ll be honest. I would not be mad if
I never have to do a green screen keying or extraction ever again. Now I get to be a person
who's doing the shot and I can help basically finalize a shot in camera. It just makes it
a more cohesive filmmaking process, and this puts us right in there next to everyone else
who's creating these shows or films.”
It was great to talk to Charmaine and learn a little bit about her work and some of the
amazing things that she’s worked on. This video is actually from a sponsor, which
is Verizon — they just turned on 5G Nationwide. So with 5G Nationwide, and in more and more
cities, you get the unprecedented performance of 5G Ultra Wideband — it’s
really fast, the world’s fastest 5G. You can download an album in a few seconds.
It’s not just gonna change your phone, it’ll change everything. I’m guessing that a lot
of creative breakthroughs will come from it too.
So this is the 5G America’s been waiting for, and it’s only from Verizon.
Verizon doesn’t directly impact our editorial, but their support makes videos like this possible.