>> Welcome to the workflow tutorial for MetaHuman Animator.
MetaHuman Animator is a feature set available in the MetaHuman
from Unreal Engine plugin.
This update requires Unreal Engine 5.2 and the latest version
of the MetaHuman rig.
We also updated the leveling phase iOS application
enabling the capture of video in-depth
through the front-facing cameras.
Today, we'll focus on streamline in Editor Workflow.
For more technical details, you can always
look at the new MetaHuman section on the Epic Developer community
website.
Let's get started.
All of the assets that we'll be showing
can be found in the MetaHuman Animator submenu
under Advanced Assets.
An additional tool, the Capture Manager,
can be found under the Tools top menu.
The data and workflow for performance capture are heavy or immediate.
If you can spare them, allocating resources to caching
can make things much smoother.
We recommend using a global cache, giving it
four threads, and at least a couple of gigabytes of system memory.
Preparing the footage for use requires
that the system has information about the device that captured it.
We added a new asset called the Capture Source.
Capture sources work with the device and calibration data.
You need one capture source for each calibration used by a given device.
IPhone devices or factory calibrated, which
means just one asset for each physical device is enough.
iPhone capture sources can work two ways.
In connected mode, we configure the IP of the device
and with labeling face active in the foreground on the device,
we could ingest directly from it.
Alternatively, and what we're using here, they can be in archive mode,
where footage has already been downloaded
and a path is set to a location containing one or more takes.
With the capture source configured, the next step
is ingesting the takes which is done from the Capture Manager.
To the left, we have at least the whole capture sources
within the project with icons indicating
the device type and status.
We can select our source and see the takes on it.
We can select takes, configure the destination path for the media
at the bottom--
which would apply to all takes selected if there were multiple--
and then add them to the queue presented to the right.
Adding to the queue will not automatically
start processing or importing.
Once everything is queued, Import All will start ingestion.
Processing large volumes of footage can take a while
and create an abundance of assets, which
is why there is an option to automatically save the new assets
as the queue moves forward.
After footage ingestion, the target folder
will contain a subfolder with a capture search name.
That's meant to prevent name clashes when multiple captures
sources contain same named takes.
Inside, we have a captured data asset and a similarly named folder.
References to data metadata are bundled together
in the capture data set.
And the folder by its side, in turn, contains the finer grain assets
produced by the ingest process.
MetaHuman faces are encoded by what we call MetaHuman DNA,
and the toolkit to calibrate it is in the MetaHuman Identity asset.
With this release, the MetaHuman is being extended to work with footage.
Components from footage will configure the componentry directly
from the capture data we choose.
And then, we can select the neutral pose
and use the Guided Workflow toolbar at the top.
We're looking for a frontal view first.
We want a relaxed expression-- open eyes staring straight ahead
and a closed mouth with the seal of the lips in view and no gaps.
All of these parameters affect the rig's rest state
in its neutral pose.
No matter the capture device class, you always need the frontal frame.
And the first frame you promote will automatically be flagged as frontal.
But this is something you can change later.
With professional grade stereo cameras,
the frontal frame alone is sufficient.
With iPhone footage, we could use just the frontal frame,
but it is better to complement it with two additional angles.
These will help with the accuracy of the calibration.
More than three frames are usually counterproductive.
You still want these frames to show a relaxed neutral pose.
But the gaze becomes irrelevant as it's derived from the frontal frame
alone.
The same markers should be visible in all three frames.
And it's best to stick to the auto-tracking set of markers that's
visible by default. The region covered by auto-tracking
is all you need for excellent performance capture.
Once your frames are tracked, you can perform
a Mesh Solve to conform the template mesh to the capture data.
To help with inspecting the fidelity of results,
we have added two new view modes.
All view modes have two configurable buffers.
But beyond the original toggle option we
had before, we have added a white mode with blank control and the duo
pane view.
Markers Display and Viewport Manipulation
are only available in the single-pane toggle mode.
But for inspection, the other two can be immensely helpful.
Next, we want to calibrate the MetaHuman DNA rig for the performer.
You can set off auto-rigging and add the result
as a new MetaHuman to your MetaHuman Creator Library as before.
Or you can choose to only retrieve the data
without encumbering your collection with every attempt you make.
This is the one instance where authentication with our back end
is necessary, so make sure that you are logged in with your Epic
and Bridge credentials.
For the mandatory steps, once free MetaHuman services
will have returned right to your project, that is it.
But an optional one is highly recommended-- adding a thief pose.
Adding a teeth pose registers the teeth position inside the head
and improves the animation solved by the MetaHuman performance asset
considerably.
Heads up, this is an operation run locally in your project.
If you added the MetaHuman to your library from the neutral pose,
that MetaHuman's bite might be subtly different from the one
in your project after you will have fit the teeth.
We can add this new pose from the Component Tree menu.
Then, look for a frontal frame to promote where
the performer is baring their teeth.
We want to clearly see the corners of two incisors and as much
of the teeth's surface as possible.
This pose only requires the one frontal frame.
And there is no reason to add more.
Most of the time, you will only see either the upper or lower incisors,
depending on the bite.
Keep the markers for the visible incisors active,
and deactivate the other two from the markers outlier.
Regardless of the teeth pose presence,
if we want to use our work to solve animation,
we need to prepare the asset for performance.
This is a computationally intensive process
and it will take a few minutes.
How long exactly depends on your artwork.
But you only need to run this once for any given performer.
Lastly, during the setup process, a skeletal measure
was created alongside our asset and configured by the authority in step.
The skeletal mesh can be useful in its own right.
But we recommend that it is not modified while being
used for performance capture work.
Duplicate it, and use the duplicate if you
wish to use it outside of the MetaHuman Animator Workflow.
We're finally at the keystone of MetaHuman Animator-- the MetaHuman
Performance asset.
And the most practical way to describe what it does
is that it takes frames of footage--
a MetaHuman rig-- calibrated to the performer in that footage
and produces key frames of facial animation.
Configuring the asset is trivially simple.
Set the footage attribute the reference capture data containing
the performance and the MetaHuman identity attribute
to reference what you calibrated on the performer that
is in that footage.
Set the start and end frames to the slice of time you want to process.
And if you want neck movement from [INAUDIBLE],,
ensure that range includes at least one frontal frame for the system
to find and use as a reference pose.
And that is it.
Everything is automated.
All you have to do from here is press the Process button.
As the footage is tracked and solved, you will see several progress
in previous stages.
And after a while, the animation will be available
for you to play and inspect in the asset.
If you're happy with the results you see in the viewport,
this is where the plugin stops and your Unreal Engine Editor comes back
into play.
You have two options to go back out to Engine.
To quickly assemble the source data together
with the results in one place, you can export the level sequence.
The option should be fairly self explanatory.
And as you can see, this gets you fairly close
to the MetaHuman performance asset viewport,
but directly inside the Engine in to level.
The other option, which is likely to be the most common in Target,
is to export the animation sequence.
Inside the MetaHuman performance asset,
the animation is mapped to the performance rig which
is a subset of the full MetaHuman rig and therefore technically different.
Don't worry, though.
The difference is purely technicality.
In reality, the rigs are semantically identical.
And export time, you can choose any MetaHuman
as a target and animation will work flawlessly on it.
We only have a couple of things left to say.
Because of the semantic parameter space of rigs,
animation will always move correctly across animated
human-compatible rigs.
And adding it to one is as simple as adding a face tracking sequencer.
SPEAKER 2: Hello, and welcome to the tutorial for MetaHuman Animator
SPEAKER 1: And a belated welcome from [INAUDIBLE],,
if anybody was wondering what the line was.
And last but not least, as first exported,
the animation will be on the Rig Logic channels, which
is great for inspection or processing raw data,
but will be frustrating for an animator to do anything useful with.
Not to worry.
At any point, animation can be back-solved to the Control Rig
face for it.
Right click the track.
Choose To Make the Control Rig, select the face for it,
pick your options, and let it work.
Now, of course, throughout the video many,
of the waits during processing and solving were sped up.
We assume you have better things to do than watching
progress bars in the tutorial.
But every single click and keystroke in the workflow
has been left unaltered.
We are absolutely positive that the ability
to do end-to-end performance capture with just 10 minutes of actual work
will lead to some amazing content.
We can't wait to see what you create with this.
Thank you very much for watching.