﻿Hello! This is Mocha Product Manager Martin Brennand and today we’re going
to show you how to take Mocha Pro’s PowerMesh Unwarp Stabilize tool
and use it as a dynamic texture to do some paint work inside Nuke.
So here’s our shot inside Mocha Pro Standalone. Keep in mind this exact same technique can
also be done in the plugin version of Mocha Pro as it has all the same tools.
We want to paint out some of the wrinkles and crows' feet around the side of this person’s face,
but as we play through the shot you can see there is some significant head turn causing a change in
the skin and how the lighting falls on their face. If we had a simple flat area of the skin we could
just do a planar perspective track, but as this is a non-planar organically moving area,
we can use the PowerMesh tracking instead to get a better result.
So first of all, we’re going to find a good frame to start our track.
With PowerMesh, it’s helpful to start on the frame you intend to start painting on
so you get the best distortion over time. So here on frame 17, their face is angled slightly
towards the camera, still showing the slight curve of the wrinkles into the eye socket and some of
the shadows falling over the forehead and side. Using the X-Spline tool, we’re going to draw
a shape around that whole area, making it a bit larger than we actually need.
In the Motion controls we’ll turn on Perspective and then Mesh,
which will make our initial PowerMesh. By default, the Mesh is set to Automatic,
which tries to find interesting features inside the shape.
If we click Adaptive Contrast and click Generate Mesh, this will do some under
the hood changes to find more details In this case however, we’re going to
select “Uniform” from our Generation Mode dropdown and regenerate the mesh. This
creates a squared set of faces which is ideal for our starting frame to paint on.
This mesh size is also a little dense, so we’ll increase the Mesh
size to about 64 pixels and generate again. Next, we will turn on our matte view and scale
down the spline a little using the transform tool. As Mocha will only do the mesh part of the track
inside the layer shape, this scale down will give the PowerMesh a border of faces
that won’t move too much, which helps localize our stabilization later to just what we need.
Finally, we’ll set the Mesh tracking options to Auto Smoothness,
as it does a pretty good job of determining how much of
the Mesh should just follow the planar track vs finding sub planar movement.
We’ll do some housekeeping and name our layer Then we can start tracking. I’ll speed up the
recording here so you don’t have to sit through it, but here I've tracked
backwards first, then returned to the frame I initialized the mesh, and tracked forwards.
So now we can play through the shot and check our result. You can see as the head turns we’re
getting some good subtle distortion into the curve of the eye socket and when he raises his brow.
Now we have what we need to generate assets for Nuke.
We’ll go to the Export Track button and choose “Alembic Mesh Data”
Note the reference frame field in the dialog is set to the frame we started the layer on.
This defines the initial state of the UV map for the mesh.
You can change this reference frame if needed.
We then save this mesh with an appropriate name to find later.
Next, we need to generate our stabilized texture for the Alembic mesh.
For this, we’ll switch to the Stabilize module. By default, the Stabilize module is going to
do our standard stabilization, locking the motion of the clip to the planar track, but
instead we want to click Mesh Warp stabilization. Just to be sure we have the right starting point,
we’ll go back to the original frame 17 and add that frame to the frame list under “Fixed frames”.
We’ll also set the quality of the Mesh Warp stabilize to “High” for the best results
and make sure we’re on “Unwarp” not “Warp” as we want to keep the distorted area in one place.
I’m going to turn off our overlays so they don’t get in the way of the
preview and then we can play through the shot to check the unwarp stabilize.
As you can see here, the ocular area on the side is now locked in place,
while the rest of the image distorts around it. Once we’re happy, we can then go back to the
start of the shot and click render forwards to begin rendering the shot.
Once the render is complete, we can then play back and review.
This generates an “Unwarp” render clip, which we can now export to disk.
Under the file menu,
we go down to “Export Rendered Clip” which brings up the export clip dialog.
We first want to choose a destination path for the image sequence.
Then, we rename the export file name to something useful and memorable.
I’m using EXR files here as that is the format of my original source clip, but
you can change this in the file format dropdown. You can also change the color space if required,
but by default it inherits the colorspace of the project.
Then we click OK and the clip will save out, showing us the progress of
the export in the top right-hand corner. Now we have everything we need to move
over to Nuke and begin setting up our paint. We’re going to need three things: Our source clip,
our exported Unwarp clip and our alembic file. When you read in the Alembic file, you can
just click “Create All in One Node” and the camera and ReadGeo nodes will be created.
While those two nodes are still selected, we’ll press tab to bring up the node search
and find the Scanline Renderer node, which will automatically connect to our two alembic pieces.
We’ll then feed the original source clip into the background input of the Scanline
Renderer node so it sits behind our mesh. If we play this back, you’ll see the grey
mesh following along with the face. If we look at the 3D representation of
the PowerMesh alembic file, you’ll see it’s a flat set of polygons with vertex distortion in X and Y.
In order for us to texture this,
we just need to feed our Unwarp render into the image input of the ReadGeo mesh node.
Here is the texture correctly positioned on the mesh in 3D.
In 2D, you should now see the clip without any mesh edges, which means the unwarped
clip is being rewarped correctly by the Alembic mesh data and projected on top.
Now we’re free to add any modifications between the Unwarped clip and the ReadGeo
to modify the textured output. In this case, we’re going to add a
RotoPaint node to clone skin from the unwarped face to replace the wrinkles near the eye.
When we select our clone brush, you may notice you can’t see the brush tools.
This may be because you’ve got the viewer node hooked up to the final output
in the Scanline Renderer. In order to see the RotoPaint overlays, just connect the
RotoPaint node directly to the Viewer node. We’ll scale up the brush slightly using
the Shift key, then using the Ctrl or Command keys and dragging, we’ll pick
areas to clone from to paint out the wrinkles. Each new stroke will make a new clone layer,
so it’s best to be as efficient as possible to avoid slowing down rendering.
I’ll speed up the paint job and we’ll continue when it’s completed. Note that i’m picking areas
that share similar small planes around the eye so I can maintain the lighting change over the shot
You can now see the change from before the paint and after.
If we scrub the shot, you’ll see that the paint only lasts one frame. In this example,
I didn’t change the life, so it was only painted on frame 17.
To fix this, simply select all the strokes in the RotoPaint node
and click one of the numbers in the “Life” column, then change the selection to “all frames”
This will now propagate the paint strokes over the entire sequence.
The advantage to using an Unwarped sequence in this scenario is that while the clip changes over
time, the same areas are being recloned per frame. This means even though the light changes across
the skin, it keeps being copied from the same flattened location, which means you don’t need
to modify or animate your paint strokes. Now we can view the result through the
Scanline Renderer node to rewarp the painted image through the Alembic mesh.
You may notice a color change in the skin, but this is because the RotoPaint node modifies
the output settings of the original image. If you want to see it combine correctly,
you can just change output from RGBA to RGB and the paint will blend correctly.
Now when we play back the shot, you’ll get a very fast preview of the result.
The reason it renders quite quickly is because we’re just applying an
animated texture to a mesh which doesn’t require much processing.
So this is a very fast way to work in Nuke for these kinds of shots.
You can see here on frame 68 that we’ve pulled in a quite dark area as his face turns.
This is easy to fix. We just go back and view the RotoPaint node on that frame, paint out the
problem area, then set the life to “all frames” again, to make sure it lasts the rest of the shot.
Now we can play back the cached preview and we’re done!
So that’s how you can do quick paint work in Nuke
by utilizing the PowerMesh workflow from Mocha Pro.
If you have any questions, leave us a comment, or pop over to the forums at borisfx.com