[MUSIC PLAYING]
MIKE SEYMOUR: We were talking earlier about subtext
and being able to convey stuff on the face
without having to just rely on the dialogue.
Vlad, I was going to ask you about the eyes.
Because it's a criticism that you
hear a lot is in this work in general,
they feel like oh, it's dead eyes,
or they're not seeing it in the eyes.
And then, of course, live action actors all
tell you that you get the soul through the eyes and stuff.
So what's your approach to eyes?
It must be something that you've had to apply a lot of attention
to.
VLADIMIR MASTILOVIC: Yeah, I think
we've made a massive improvement with MetaHumans.
Going back a few years, shaders for the eyes in Unreal Engine
have improved tremendously, supporting things
like refraction, which until some point
weren't imaginable in real-time engines.
And then, of course, all of our solutions are scan-based.
We sort of scrutinize fitting of the meshes
to eyes under extreme detail, checking reprojections
from multiple camera views for every single expression,
and then converting that into a biomechanical model that
sort of respects the cornea bulge,
and calibrates the movement of the flesh
in all different directions.
Having said that, there's certainly still
a way to go to further improve.
But looking into what people are doing with sample assets
that we have released a few weeks ago,
I'm really encouraged by the visuals
that I see rendered by individuals that didn't
spend much time doing them.
But you can really see like strong creatives
and strong artists are going to be
able to produce completely new kind of media now,
thanks to this.
MIKE SEYMOUR: Jerome, how did you
find the eyes and the characters that you
were doing when you were doing it remotely obviously.
You then got the assets.
Did they work for you?
Do you have a comment on it?
JEROME CHEN: I don't know what makes eyes work.
[LAUGHS] It's impossible, it's impossible.
I've been trying to figure that out
for like, 20 years or longer.
They're better.
I found myself making less comments
about eyes, like, a lot less comments,
and more about whether you believe
what that person is doing at the time,
just in terms of their overall performance.
I mean, there's just been so much done in terms of shading.
Shading has helped a lot.
In the newer--
I think global illumination in general and ray tracing
has helped realize eyes.
In the old days, the rendering eyes
were very difficult to create believable eyes.
But I think once the GI rendering
technology is more prevalent.
That's helped.
That's just been better universally.
Just by going to movies, you see how many digital characters
are out there, and you don't really
notice the bad eyes anymore.
They show up occasionally.
If that's the case, you basically
pan the camera off the character or do a camera shake or a lens
flare right across the face or something to hide it.
[LAUGHS]
VLADIMIR MASTILOVIC: So what I really find fascinating
about the eyes is the fact that we hold 90% of our resolution
is just about two degrees field of view,
and the rest is just a blur.
And our mind has a prediction system
which understands which is the next target--
which is the next blurry target that is relevant to me.
What I also find is interesting is
that we have very specific patterns
that we run when we look at somebody's face.
And it's dependent on the emotion
that that face portrays.
So depending on whether we're sad, or angry, or just idle,
we have a pattern that will basically track
various positions in our face.
So the believability of the eyes is
going to be a lot in the animation itself,
as well as the scene integration, which
is what I found, especially when you
have a lot of characters in the scene,
I think that that's really often the crucial bit.
How do you integrate all of these characters in the scene,
and where does the eye gaze go?
ISAAC BRATZEL: Sorry.
Just to touch on that, as well, I
think the eyes are so complex on every level,
from the animation to the character,
to the actual geometry.
It's such a complex geometry that in order
to have shaders that actually work well and represent that,
that geometry has to be accurate to real world things.
And in fact, that's what we're getting better at.
Because you can't really use refractive shaders
and everything else if that shape
of the cornea and the lacrimal fluid and stuff
isn't there because it's not going to react properly.
So I think that's something that I think
we're getting to the point now where you can see how it's
progressing, and that will get there gradually as rendering
engines improve, as shaders improve,
and as the geometry that we capture
is actually more accurate.
JEROME CHEN: And also, we're talking
about the shape of the eye itself,
and you're touching upon it a little bit,
it's the shape of the eyebrow-- the eyelids
that form, you know, the shape of your eye
and how much tension is in that eyelid,
and if you're doing your saccades,
as Vlad is mentioning.
If your eye shifts, the eyelid has to bulge in the right way.
And while it's doing that, you may be squinting.
So as you know, we look at eyes all the time.
It's the first thing typically a human looks at.
And those thousands of details are just,
we're just trying to check those off the list as we
progress in the creation of these digital humans.
But we're learning more.
What's important, though, is that now you're
able to take that knowledge, gather data from it,
and recreate it.
And we're getting better at that.