Narrator: In a lot of the behind-the-scenes shots
from "Avatar: The Way of Water," you'll see these monitors.
Zoom in and you'll see the performance
of the actor playing Quaritch, a 9-foot-tall Na'vi.
The robotic monitor was a reference
for the actor in the scene
and for the VFX team who added the CG giant in post.
It was a huge step forward from the first "Avatar,"
which only had a handful of scenes
with humans and Na'vi interacting compared to the sequel.
Filmmakers have been making actors
taller or smaller than they really are
for over a century.
But making sure the scale is realistic
without ruining the illusion
is where the real work begins.
And one of the pioneers in the space is Wētā,
the visual effects company
that's overseen some of the most groundbreaking
CG characters of all sizes,
in movies like "The Lord of the Rings"
and "The Way of Water."
Eric: We always have to play games in film
to make that feel
and actually be able to get good performances
from the different scale that you want.
If they're 6-foot, the tree's going to need to be
30% smaller if the characters are going to be 30% bigger.
So there's a lot of planning involved there.
Narrator: It's a puzzle filmmakers have been trying to solve
since the invention of film.
Take this shot of 1909's "Princess Nicotine."
The tiny fairy next to this normal-sized human
was actually standing on a platform far behind the table.
This is known as forced perspective,
in which one performer in frame
will place themselves closer to the camera
and the other farther away.
Forced perspective requires a sharp depth of field
so both the foreground and background
appear equally in focus.
And that means a lot of light.
Visual effects artist Peter Ellenshaw
said that when working on 1959's
"Darby O'Gill and the Little People,"
the production used so many lights
for the forced-perspective shots
that it blew out the circuit breakers
at a nearby power station.
Forced perspective works
when the characters are separated.
When they have to interact,
things get more complicated.
This was an issue Peter Jackson encountered
when he started shooting
"The Lord of the Rings: The Fellowship of the Ring"
with wizards, dwarves, and hobbits.
To expand how scale could be used in these movies,
the crew had to manipulate their sets and props
while filming at just the right angle.
Here you can see Gandalf and Frodo
sitting side by side.
All right, then, keep your secrets.
Narrator: But on set, Elijah Wood was sitting
on a shorter stool pushed farther back,
while Ian McKellen sat right in front of the camera.
These sorts of raised sets
are a common forced-perspective technique
that you can also see two years later in "Elf."
And forced perspective also needs to be paired
with props that are made to scale.
In this shot, Frodo and Gandalf were actually
sitting at a table that had been split apart.
On one side were normal-sized props for Frodo.
And for Gandalf, a small-scale table
with a mug that was two-thirds the size.
The reason these forced-prospective shots work
is that the camera stays still.
Most of the camera moves were quite locked off.
Narrator: Once the camera moves, the illusion is ruined.
Eric: It screwed with your head a little bit
when you were on set,
because when you watched it from the side,
it didn't look right at all.
Narrator: To accomplish this moving forced-perspective shot,
the crew placed the table on a moving platform
and programmed the camera
to move in the opposite direction.
They programmed the camera's moves in advance,
making them easy to replicate.
That way, in scenes like this one,
they could shoot the actors separately
in the exact same way
and seamlessly combine them.
The shorter Bilbo could walk in front of
the much taller Gandalf,
and it could all be combined
with the two versions of Bilbo's house:
one hobbit-sized
and one made 33% smaller for Gandalf.
Oh!
Narrator: And these shots could consist
of more than two characters at once.
Nine companions.
Narrator: To get all four hobbits, two humans,
a dwarf, an elf, and a wizard into the same shot,
the crew filmed scale composites of each actor
against a blue screen
with the proper framing and camera angle
for the characters' heights.
The images were then composited together.
This technology continued to be perfected and used
to expand the "Lord of the Rings" universe.
But it hit a snag as Wētā worked on "Avatar"
and "The Hobbit: An Unexpected Journey,"
two movies shot in 3D.
Eric: A lot of the cheats
we were able to get away with before,
we can't get away with as much now.
So instead of shooting with one camera,
you're shooting with side-by-side cameras,
so you get proper 3D.
The false perspective really falls apart quite quickly,
because you understand the depth of the shots.
The motion control really kicked into gear
for the "Hobbit" trilogy.
Narrator: At this point, the cameras were much more precise
and reliable than they were in the original trilogy.
Like during this sequence.
Eric: It got the results you needed.
You were able to do camera moves you wanted,
you were able to move around,
do the different things you had to do to get the shots.
Narrator: But it can pose a challenge
when the actors are playing non-CG characters.
Because the actors were frequently shot in different takes
to account for the differing heights,
they weren't able to directly interact with one another.
It wasn't very actor-friendly.
Narrator: Size manipulation is a lot easier
if the character is fully CG,
like the giant in "The BFG,"
played by Mark Rylance.
Mark could be on set with actor Ruby Barnhill,
as long as he acted above her on this platform.
But even this movie innovated new riffs on past techniques.
For example, Ruby was surrounded by oversized props
when her character was in the giant's house.
But unlike past large-scale props,
some of them were blue.
That way, the animators could add movements
to this giant-sized squash.
Some other tricks to maintain proper eyeline
and sense of scale are low tech but effective.
In the "Avengers" films,
Josh Brolin plays the 8-foot-tall Thanos.
The mo-cap sensors he wore were so good
at picking up subtle expressions and movements
that his real height didn't matter
until he was interacting directly with other characters.
So to fill out the height gap in this shot,
Josh wore a cutout on top of his head
so Robert Downey Jr. knew exactly where to look.
You can also see the 5'4" Tatiana Maslany
wearing a cutout here
to look like she's the 6'7" She-Hulk.
Just like in "The BFG,"
she stood on an elevated platform in these shots
and occasionally sat on a cushion.
But eyeline isn't the only issue you need to resolve
when making a performer taller.
In this walking shot,
She-Hulk isn't standing next to anybody,
but Wētā found another issue.
Tatiana's real stride didn't match up
with She-Hulk's much longer legs.
Where Tatiana would take 10 steps,
She-Hulk would take seven.
That's where retargeting, or character mapping, comes in.
This is when the VFX artists
take all the movements and expressions
from the performance-capture cameras
and apply them to the animated character.
When the character is a different size,
Wētā's software Nuance needs to fill in the gaps,
like in translating actor Sam Worthington's posture
over his Na'vi body in "Avatar"
or turning Andy Serkis into Caesar
in the "Planet of the Apes" trilogy.
If he's quadrupedal, you want to make sure
that what is absolute here
from his performance is his two feet
and the markers on the end of the arm extension.
Narrator: The crew worked with actor Malia Arrayah
to retarget She-Hulk's stride.
Malia could serve as a more height-appropriate stand-in.
But this isn't a new practice.
In the past, stand-ins were used in shots
where you couldn't see their face.
By having Malia's movements recorded on a mo-cap stage,
VFX artists could map her stride,
along with Tatiana's facial capture,
onto She-Hulk.
Everything about you is just working.
Thank you!
Narrator: The better CG and camera tech has gotten,
the more seamless these scale shots can be.
For all of the technological breakthroughs
in the original "Avatar,"
there are actually very few shots
of humans and Na'vi characters together,
as the process at the time was much less precise.
Take it easy. Don't get ahead of yourself.
Narrator: For instance, the shot of Neytiri
holding Jake Sully's human body
was achieved by two green-suit performers
approximating Na'vi size.
But that new eyeline system
meant there could be many more shots
of humans and Na'vi together.
It consisted of a monitor playing an actor's
previously recorded performance.
That monitor was attached
to overhead cables at four points.
So whenever Scoresby had a scene with Quaritch,
the monitor would play back Stephen Lang's performance
at the proper height.
It got rotations, it got height,
so as he moved around, it knew exactly
where his head was supposed to be,
his eyes were supposed to be,
and it moved around set.
Narrator: Take this shot, where Quaritch walks alongside
two human-sized characters.
We had a live-action character, Kevin Dorman,
actually walking beside them.
Kevin Dorman is the easy one, right?
He's just live-action character
walking on a construction set.
But he has to match the speed of Slang,
which was performance capture.
So we take the performance capture,
and we put it on the cable cam, the eyeline system,
and we actually ran that down the construction set,
and Kevin Dorman was able to walk along with him,
and he had an eyeline,
so he knew the speed he was supposed to walk.
Narrator: But it wasn't just the actors
who needed precise references.
Real-time depth compositing
allowed James Cameron to see the CG performances
as the footage was being captured.
By seeing a rough version
of what the finalized CG shot would look like,
the director could better set up shots
of CG and live-action characters
without worrying about a single camera move
ruining the illusion.
Wētā also used real-time compositing on the first "Avatar,"
but the traditional combining of CG and live-action footage
is typically done in layers.
Eric: Back in the day, you sort of had an A over B,
which meant you had either the live action or the CG,
and it was one over the other.
Narrator: The new system built for "The Way of Water"
looked at each shot's elements in pixels instead of layers.
Eric: We wanted to place the depth of each pixel
as we were shooting,
so that instead of doing A over B,
we could take each point on space and say,
you are either in front or behind the other character.
Narrator: And that helps achieve the correct scale.
Eric: You could actually see how big someone was,
because otherwise you're just getting that fake perspective,
even inadvertently,
because a character is always over the top of one another.
Narrator: Thanks to all of Wētā's innovations,
making a character look bigger or smaller
is no longer a guessing game.
Those perspectives can be less forced.
But even in the completely CG Pandora,
some of the most convincing scale illusions are tangible,
like when Quaritch holds up his own human-sized skull.
On set there was an actual prop skull,
but scaled down from human size.
That way, the actor's hand looked Na'vi-sized.
Eric: When he crushes it,
it feels like a normal-sized skull,
because you never think about
his hand being a small, normal-sized hand.
You just assume it's the Na'vi's hand crushing it,
because you go from a shot of Slang to a hand,
and your brain just fills it in as just a big hand
instead of a painted smaller hand.