motion tracking is an essential aspect of many vfx shots whether you're
turning your actor into a cyborg or adding a computer-generated prop into your actor's hand
hello everyone i'm zach and welcome back to another tutorial we often use blender to
complement our hitfilm vfx tutorials and tracking something in the scene is usually the first step
today we'll be taking a detailed look at 3d tracking in blender so you can create some
cool vfx to be composited in hitfilm when you record your video footage make sure the
object you want to track has some trackable features when tracking someone's face we like
to draw these tracking dots we just need some sort of detail that the tracker can lock onto
before we even open blender it's a good idea to process our video footage before tracking it
this clip was shot in log so it looks very flat i'll bring the video into hitfilm
then i'll add a curves effect to bring some contrast back into the image
i'll add a little more contrast increase the saturation and add some sharpening this makes
our video easier to see and can help the tracking process lock onto the details better now we can
export our video as a png image sequence make sure to create a new folder for all the images your
footage doesn't have to be an image sequence to work in blender but i find that it works the best
open up blender and create a new file using the vfx template this sets up our workspace
with everything we need for motion tracking click open and navigate to your video clip to load it in
over in the output properties let's set our resolution and frame rate to match the video file
click set scene frames to make your blender project the same duration as your video
then click the prefetch button this will load your footage into memory so it can play back smoothly
our footage is all set up in blender so let's begin tracking
i'll start with the default tracking settings ctrl-click to add a tracker
then open up the track tab so we can better monitor the tracking process
the e key brings up the tracking menu so let's track forward the track stop before reaching
the end of the shot we can reposition the tracker on its last good frame and track again from there
nice it made it to the end and it seems to be sticking well press ctrl l to lock it preventing
any accidental adjustments the next track goes all the way to the end if you find these motion
trails are getting in the way you can go to the clip display menu and hide the track paths
this next track stops early so i'll delete it by pressing x let's see what settings we can adjust
to make it track the whole shot normalize can sometimes help if the lighting in your footage
changes over time setting the match type to previous frame can really help if the shape
of your tracking markers change over time now the tracker makes it all the way to the end
and it looks good pattern size and search size are the same as the red and green boxes when
motion tracking in hitfilm you can display the search box from the clip display menu
if the motion you are tracking is very fast you may need to increase the search size
these tracking settings only apply to the trackers we add next not the tracker we
currently have selected you can adjust its settings from the menu on the other side
while i'm tracking i don't really watch the main viewport i pay more attention to the small track
window which stays centered on the tracker making it easy to see if it drifts off course
i want this tracker to be a bit bigger instead of adjusting the pattern and search size you
can scale it by hitting s this is good for a few select trackers but if you find that
they all need to be a different size you should change the settings beforehand i've gone ahead
and tracked all the easy points that are always visible the rest of the markers disappear and
then reappear later on when i track this one it goes to the end but doesn't do a good job as it
starts sliding off the marker find the last frame where it's still good then press e and clear the
track path to disable the tracker before it goes wonky if we deselect all with alt a the tracker is
still visible while disabled you can hide the disabled trackers from the clip display menu
here i have two separate tracks one that was tracked forward for the start and one that was
tracked backwards for the end but they are on the same marker with the one selected we
can shift select the other then go to track join tracks now they have been combined into one track
that starts active disables then reactivates cool repeat this process to track all the other markers
before solving and turning these 2d tracks into a 3d object i like to adjust the 3d camera right now
the camera position and orientation doesn't make sense compared to our footage i'll add a layout
workspace to make it easier to work in 3d then i'll select the camera and adjust its location
and rotation to roughly match the camera we filmed with there much better to solve our object we need
at least 8 trackers at all times we have a lot more than 8 in this case so that leaves room to
delete some trackers if they are lower quality or causing problems later on if we look at our
objects we've actually added all these trackers to a camera but we're not doing a camera solve
select the camera then press a to select all the trackers unlock them with alt l then copy them
click the plus button to add an object paste the trackers and lock them again then delete them from
the camera of course you could just add an object first and avoid this cutting and pacing step
if you know the sensor size of your camera and the focal length of your lens you should enter them in
here the solve will still work without providing this information but it's an easy step for more
accuracy now we're ready to solve our object go to the solve tab and click solve object motion
we can see that our solve error is 0.53 pixels which is quite good anything less than 1 is
a pretty good place to be you can reduce the solve error by setting the keyframes
find the frames that show the most parallax between points and that best represent the
motion of the object i'll set mine to the frames before and after this rotation in this case
the error is the same but in other projects using good keyframes made a huge improvement
another way to reduce the error is to delete trackers with a higher error
in the clip display menu you can enable info which shows the average error on the selected tracker
delete tracks with errors much higher than the rest this one on the eyebrow likely has
a higher error because the eyebrow moves independently from the head
deleting that tracker and solving again or error is reduced nice
with our solve looking good we can now apply it to a 3d object add a camera solver constraint to the
camera even though we aren't tracking any camera movement this is needed to show the tracking marks
in the 3d viewport now we can see where the object is in 3d space relative to the camera
adjust the object's scale setting to place it the right distance away since i placed my camera two
meters away from the origin and kirstie was about two meters away from the camera while filming
i know my object should be roughly at the world center in the geometry drop down click 3d markers
to mesh this creates a 3d object representing the track points you can hide the tracker overlay from
the viewport now if it's in the way looking at the object in 3d it seems to make sense as the points
match the curvature of a human face this is a good sign if your points don't make sense you may need
to improve your solve click the set as background button to see your footage through the 3d camera
i'll increase their opacity to make them easier to see
our object still doesn't follow the tracked movement add an object solver constraint
select the camera and object then click set inverse now we can see our 3d object moving
and rotating in 3d space and the movement looks good and makes sense based on the footage cool
let's add a custom 3d model into the shot to make use of the tracking data
this iron man helmet model is free to download from blindswap.com
link in the description since it's a blender file we need to append it instead of importing
the model comes in with some rotation animation so i'll clear those keyframes
we now need to position scale and rotate the model to line it up with the tracking markers
and our video going into wireframe view can help with the alignment process with the model
roughly lined up shift select attract object and parent the model to the track with control p
now we can go throughout the shot and refine the placement of the model we can see that the
model is tracked in nicely but some of kirsty's ear hair and hat stick out around the helmet
that can be easily fixed in hitfilm with some simple compositing after we export out a blender
once you are happy with your blender animation you can bring it over to hitfilm to create a
final vfx shot if you're using hitfilm express without the 3d model add-on you can render an
image sequence out of blender to be combined with your footage and hitfilm this 3d model came with
some materials already set up so you just need to add some lighting before you render check out the
tutorial linked in the card on screen to learn more about using blender with hitfilm express
if you want even more control over your 3d scene without having to re-render out of blender
you can export the 3d animation over to hitfilm you'll first need to install the free blender to
hitfilm exporter linked in the video description with a track object selected you can export to
a hitfilm composite shot this will bring our 3d camera and our 3d object motion data into hitfilm
next we need to export the 3d model itself i'll make sure the iron man
helmet objects are selectable then with the object selected export as an obj file
make sure to check selection only so no unwanted objects are included
open up hitfilm and import the composite shot bringing in the camera and track 3d point
then import the 3d model you can adjust the materials now or later but make sure to enable
any animation groups if you plan to animate separate parts of the model in hitfilm for more
info on importing 3d models into hitfilm check out the tutorial linked in the card on screen
make sure to uncheck center anchor point and auto normalize scale to ensure that the 3d model will
line up with our 3d scene properly drag the model onto the timeline and parent it to their 3d point
if we drag in our footage underneath everything seems to line up nicely
cool we've successfully brought our tracked object from blender into hitfilm
to remove and clean up the areas outside the helmet you'll need a clean plate meaning footage
without the actor in the frame our footage was longer than the final clip and since kirsty moved
during the shot i was able to extract enough of the background behind her without needing
a separate clean plate i then applied masks to these exported frames to cover the unwanted areas
now it looks like kirsty is wearing the helmet instead of it being stuck over top
to take this shot to the next level you can set up proper materials and lighting on the 3d model
you can also add additional effects like glow on the eyes and can blend the digital model with a
video footage with color correction blur and green tie the whole shot together and finish
it off with a color grade a solid 3d track from blender combined with compositing and effects
in hitfilm can result in some epic v effects check out these other tutorials to learn more
about the hitfilm and blender combination leave a comment and hit the like button if
you learned something new remember to subscribe for more tutorials like this see you next time