Ugh, would you calm down?
Now read my lips.
We gotta talk.
Narrator: One of the hardest jobs in visual effects
is making animals look realistic when they talk.
It's a challenge effects studio Weta Digital has taken on.
And if you look at Weta's talking fox
from the 2021 movie "The Green Knight,"
you can see just how far we've come.
But getting from here ...
Don: [neighs] Oh, my. Can't ask him that.
Narrator: To here ...
Fox: To end his song as he saw fit.
Narrator: Wasn't easy.
When the whole CG-talking-animal genre was born in the '90s,
filmmakers faced one overarching challenge:
how do you make a fundamentally
unbelievable thing believable?
Before, movies would just dub over the mouth movements
of live animal actors or animatronic puppets
and later moved to compositing footage of live animals
with 3D digital models of the animals' mouths.
Over the years, studios like Weta have mastered the art
of creating fully digital creatures that speak
and express themselves in naturalistic ways.
Mike was one of the lead animators
on "Rise of the Planet of the Apes,"
where the chimpanzee Caesar slowly learns
to converse with humans.
Caesar: No!
Narrator: Human actors not only voiced
but actually played the apes,
filming in motion-capture suits
that tracked their movements.
Weta's animators then had to map those performances
onto the digital ape models,
making the apes deliver their lines
and emote in ways that stayed true to the actors' choices
without feeling too human.
The very different anatomies of humans and apes
often made it impossible
to exactly translate those acting choices
one-for-one in dialogue scenes.
For example, apes have heavier brows
and larger neck muscles,
which had to move differently when the apes were speaking.
Making sure that we've got the wrinkling working well,
how the muzzle inflates,
how still he is when he says his line,
sometimes that isn't a dead translation.
Narrator: Nowhere are the anatomical differences
more extreme than in that mouth area,
which is obviously crucial for dialogue.
The muzzle on a chimpanzee, for example,
is a lot more malleable and dexterous than a human mouth.
The apes have the ability to move their lips
in such big ways compared to humans.
Narrator: That meant that in some cases,
animators had to blend some of the actors' human lip shapes
onto the digital facial puppets of the apes,
like when pronouncing the phoneme "sh."
A chimp's lips extend into a long pucker to make this shape,
which would distract from the actual dialogue.
So Weta reined that in, tightening the mouth shape
and keeping a fuller, more human lip.
These lip and snout differences
between humans and apes actually made Andy Serkis
a perfect casting choice for Caesar,
since he often articulates with his lips
while delivering lines.
Andy can be an overperformer.
He started in early days of motion capture,
when the fidelity wasn't as good.
It didn't capture as much motion,
so when you see him perform for Caesar,
he does move his lips a little bit more
than you typically expect
when you actually talk to him in person.
Narrator: Beyond the most visible parts of the mouth,
Weta spends a lot of time looking
at dental scans of actors' teeth
and videos showing how the tongue moves inside the mouth.
All that internal structure inevitably drives
how the surface of the mouth behaves,
how the body behaves.
Apes and any other character or animal we're doing,
we can actually take the geometry of their mouth,
their teeth, their gums, their tongue,
so that the same way we know
how lips move over a human's skeleton,
the same thing's going to happen with a dog, with an ape.
Narrator: That close understanding of mouth anatomy
came into play on "The Jungle Book,"
where Weta had to make the giant orangutan King Louie
not only talk, but sing.
Animators had to figure out just how far
to hold Louie's tongue back in his mouth
when he sang his lines.
♪ You'll see it's true ♪
♪ Someone like me ♪
In the end, though, animation supervisor Paul Story
said his team had to sell the singing performance
with body motion first,
like by carefully timing Louie's chest movements
to show him breathing in between lines of song
and by coordinating his body movements
with every little head motion,
a lot of which was based on videos
of Christopher Walken reading his lines.
When Weta made dogs talk and sing
for another live-action remake, "Lady and the Tramp,"
animators kept an even lighter touch on the mouth area.
The difference between ape's mouth and a dog's mouth is,
you know, a dog's got a long muzzle,
and as soon as you start doing too much with the jaw,
it suddenly becomes this overanimated performance,
and if you've got really fast dialogue
or somebody singing a song, it quickly falls apart.
The thing that we found with dogs
and the fox in particular was
if we pushed the performance out to the front of the mouth
and then supported that performance in the rest of the face
and all the way down into the body,
I think that really kind of grounds it.
Lady: Well, this is goodbye, I think.
Tramp: Yep.
Lady: Bye. Tramp: Bye.
Narrator: This approach to the fox in "The Green Knight"
helped guard the mystery around him.
Since he talks in what feels like a dream state,
director David Lowery wanted to keep viewers guessing.
Fox: Are you this man?
Mike: The dialogue was almost like,
wait, did I just see that?
Is that talking?
David wanted to keep it so subtle
and such a character that you never really understood.
Narrator: So the animators focused
on making the fox's breath
move through his body as he speaks
and on nailing the subtle movements
in the nose, neck, and jowls.
They even zoomed into parts of the body
you wouldn't think of as being tied to speech,
like timing the fox's vocal delivery
with his footsteps and the twitching of his ears.
They're all folding into that little bit of dialogue,
so the dialogue may be subtle on the front of the muzzle,
but the whole character is speaking.
Narrator: Including, maybe most importantly,
through the eyes.
We've always known that the eyes are where you get
a lot of the performance out of the characters.
Narrator: That's why Weta's always improving the way
it models its eyes, including parts
that we don't even see on camera.
Eric: If you go back to Gollum, the eyes we had,
it was three pieces of geometry
with painted textures on them.
Our eyes have so much depth in them now
that you can actually fly right into them.
We actually put all of the proper strings and muscles in,
connecting the pupil and generating the iris, everything.
We've talked to eye doctors.
We've got eye scans.
Narrator: Weta's animators have found
that for dialogue scenes, nailing eye focus is key.
And so we're really specific about making sure
that there's something, even in the digital space,
the off-camera, that isn't even going to be in the shot
that the characters are focusing on.
Narrator: The studio's also learned
that our eyes are constantly making tiny movements
to take in new pieces of a scene,
even when we appear to be holding a steady gaze.
There's this extra little bit of motion
that sits on top of all that detailed performance.
Narrator: For emotional dialogue moments like this one,
the team consulted with an eye surgeon
to learn how moisture beads up on the eyeballs
and runs into the fold
between the lower lid and the eyeball.
The blood flow as someone's crying,
you get that actual reddening in the caruncula.
You really feel that emotion come through
with all that extra detail.
Narrator: That obsessive level of detail
has made it possible to branch out
into new types of talking animals,
even the swimming kind,
in season two of "The Umbrella Academy."
After creating the chimp Pogo for the Netflix series,
Weta was recruited to build a talking goldfish.
AJ: Your recovery has been an inspiration to us all.
Narrator: This character, AJ Carmichael,
was a new sort of challenge,
according to animation supervisor Aidan Martin,
since normal fish don't even have tongues
and there aren't many parts of a fish's face
that would articulate the way we see with human speech.
But Weta's animators found a solution:
adding controls to AJ's lips and mouth
so they could overlap,
allowing a wider range of mouth shapes
as he delivers his lines.
It's all about hitting that sweet spot
between what's anatomically possible
and what looks credible on screen.
Weta's mastered that balance
to create talking animal characters
that are every bit as expressive and believable
as their human counterparts.
Caesar: Do not come back.