>> Since we are talking about virtual production,
we thought we would show you some of the things
we are doing down here because we are in the middle
of a virtual production in-camera VFX shoot
using Unreal Engine to drive these LED walls.
We have got a few different scenes we are shooting,
so let's go down and take a look around.
So the operator would be here
actually controlling Unreal Engine,
so all the information that we are talking about,
moving the screens around, the tracking of the cameras,
the information from the focus of the camera,
that all comes, gets funneled into Unreal Engine,
and then the operator is here,
and they can control the whole scene in real time
because it is all being rendered in real time,
so if the DOP is like, "Oh, I need more light here,
or let's rotate the world slightly.
Let's get the Sun coming in
from a slightly different angle",
that can all happen here in real time.
We have been working with the virtual production tools in Epic
for years now, I think, in terms of whether
that is Virtual Camera on lookup stage or iPad cameras
with ARKit or performance capture coming out the engine.
We have been exploring those things.
I think one of the successes of an in-camera VFX shoot
like this is preparation
and going into it in previsualization
and knowing what you are going to shoot so that ...
When we built this,
we considered making a static and LED wall configuration,
but it made far more sense to have everything flexible
because we can move the vehicles.
We can move the screens. We can move them up.
They can move up and down on motors.
Everything is completely configurable,
and because we have the QR code system,
which lets us track where all the screens are anytime,
we can have that dynamic movement,
and we can give the director
and the creatives any sort of control over
how they want to change it on the day
rather than it being a fixed setup.
We are testing some new virtual production technologies,
some of which will be in 4.26 of it.
We are working closely with Epic
to test those out, give them feedback,
explore them, help them, develop them.
We have done a lot of projects over the years,
and each time we do,
we try and push real-time technology in some way.
We work as partners and push that tech,
and we had the small thing,
and we launched ARKit with Apple.
We did that with Epic, and then we have done live events like
[Indistinct]
through to this virtual production
thing we are doing here.
4.26 has a lot of cool new features in it,
and we have been exploring hair and fur in that system,
and, well, it has got a long history of you
creating hairy creatures from the Apes franchise,
"Umbrella Academy", a lot of other ones,
and we have been wanting to say, "How far could you take
a movie-quality hair-and-fur system
and push that straight data into the engine
and see what happens in real time?"
>> Hello. My name is Keith Miller.
I am the Creative Director and Visual Effects Supervisor
for the Special Projects team here at WETA Digital,
and today I would like to give you
a little bit of background on a project
that we have been working on
and we would like to share with you today.
So at WETA, we have got a pretty long history
of working with filmmakers and helping them
to tell their stories through visual effects,
so we develop characters. We build worlds.
We work to advance the technology as required
to support that storytelling and that filmmaking process.
What we have been starting to see over the last few years
really is real-time technology
increasingly becoming a part of the filmmaking process,
and that is across the board really,
everything from previs through to final pixels in some cases.
So when we think about Unreal Engine,
we approach it as a framework,
not only for background in-camera content
during production as Alasdair just showed us,
but for producing final pixels for hero animated content,
so we wanted to push beyond the static backgrounds
that we are used to seeing for kind of modern in-camera
LED-wall productions and really see what is possible
when working with animated content and characters.
So we put together a small in-house team
to explore this idea and create a fun little animated short
that we are excited to present for you now.
So hope you enjoyed our short.
I know we certainly enjoyed exploring new workflows
on the project. Traditionally for us at WETA,
we are used to adapting feature-film VFX content
for real-time use cases, so that is VR, AR games,
or maybe even proxies for our own real-time renderer
for lighting and animation purposes.
That process of optimizing for real time,
it is constrained when the fidelity is driven
by the performance requirements of the target systems,
so we were keen to explore working the other way around
from that traditional film-to-real-time scenario.
So we want to ask the question,
what if we were directing content purely for a scenario
where we can create assets from scratch
for production of linear narrative media?
Are there real-time workflows that we can leverage
for producing rendered media in that fashion?
Can we work in real time
and take advantage of those processes
without having to overly worry about performance throughout
and then still render to create pretty pictures?
So let's jump in and take a quick look
behind the scenes in a couple areas.
So one area we were particularly excited to explore
was the strand-based hair system that was introduced in 4.24,
so we started working pretty closely with Epic end of 2019
where we are using some existing ape assets
to push the integration of what is proprietary for our system
with the new strand-based system,
so we helped drive the groom requirements,
just generally make sure that the systems were able
to handle the complexity
of a heavily furred feature-film VFX asset,
so we continued that work into this year with our meerkat
and our eagle on this project,
took some trips to the zoo for our reference,
began to study their fur in detail,
so our meerkat was initially groomed at real-world densities
and widths with enough segments along the strands
to match the actual curvatures
that we would see in the reference,
and then, from there, there was some optimization
to manage the real-time constraints,
but as you can see here,
we are able to get pretty solid performance
with a dense groom in the engine.
This meerkat groom is, I think,
somewhere around 400,000 strands.
So the feathers were particularly challenging for us
in real time primarily
due to the anatomical structure of a feather
and some of the rigging requirements
but also just due to the general density required,
and just to confirm,
these feathers are all constructed from strands.
They are not using cards or anything like that.
So the eagle groom that you can see here
is somewhere around 2.5 million strands,
and the shaft of each feather is geometry to allow for rigging,
and then the barbs on each feather are constructed
from strands that are grown along the shaft or the rachis.
As much as possible, the shaders were built
to mimic the same biologic principles
that govern real hair and feathers,
studying the growth patterns,
the proteins that are in charge of the pigmentation
and the release-timing mechanisms.
That is all key to replicating the patterns
that we see in specimens of meerkat fur
and the martial-eagle feathers,
and all of the banding and patterning
that you can see here
was handled procedurally in the shaders
to replicate those processes.
So another tool we explored during production
was Unreal's Live Link, and while we had already been
using that in production to some degree
for bringing performance capture into the engine,
this was the first time that we used it with animators
working inside Maya.
That is really important for them to be able to evaluate
their posing in the engine under the actual shot lighting
and with the fur volumes in place,
and that is really helpful for them
since they can get an understanding of how the shapes
and the volumes read since it is something
that is normally much slower to iterate on and refine.
The project itself was actually quite performant
and able to run at blazing speeds on the NVIDIA 3090
that I have in this machine, and you can see me here
just kind of playing it down in the Sequencer,
and the frame rates are rather high
even with all the fur in place,
but even as performant as it was,
we knew we wanted to leverage everything we could throw at it
for final quality renders,
and that way we could take advantage of large numbers
of temporal samples and really crank up the detail
by controlling a number of the variables at render time.
So we have wired up some quality switches like this one
where we could force a LOD on the environment for renders
and use a crazy amount of geometry for detail
without impacting our interactive workflows
in any way. We are really excited
that we were able to share this work with you today
and are looking forward to continue to push the technology
and see how far we can take this.
Thank you.