Hello everyone, I’m Zach and welcome back to another HitFilm tutorial. Today I’ll
be showing you how to use Foundry's CameraTracker to place digital elements into your video
footage. This tutorial focuses on the camera tracking portion of the process, rather than
the realistic compositing. Check the description for some tutorials on integrating digital
assets into your video realistically.
Foundry's CameraTracker was added in HitFilm version 13 and is exclusive to HitFilm Pro.
The basic workflow to track the camera movement in your footage is to Track Features, Solve
the Camera, and Create the Scene. The CameraTracker includes various settings to fine tune the
results and produce a high quality track, so don’t go away if you want to level up
your tracking skills.
The CameraTracker works by identifying many small features within your video, and following
their movement over time. Based on the movement of these features, and how features closer
to the camera move differently to those farther away, the Tracker is able to calculate or
“solve” a 3D camera.
If you are looking for an even quicker and easier way to achieve a 3D camera track, be
sure to check out our very own CamTrackAR app, available for free on the iOS App Store.
This app allows you to record your video footage and 3D camera data at the same time, preventing
the need for any camera tracking in post production.
Here’s the main clip I’ll be tracking. There are many uses for a 3D camera track.
We could place a billboard on this building, add a grunge texture to this road, add some
floating text, or even add a 3D model. I’ll start by making this clip into a Composite
Shot, and I’ll add the CameraTracker effect. If your footage has something moving differently
to the camera, like the flags in this clip here, it’s a good idea to make the Tracker
ignore that movement by using a Matte. We only want the CameraTracker to look at features
that are static. Add a new white Plane layer, and hide it from view. With the Plane layer
selected, use the Freehand Mask tool to draw a shape around the moving object. Activate
keyframing for the mask Path, then scrub the timeline, adjusting the mask points throughout
the whole shot. With our mask finished, make this layer into a Composite Shot, making sure
to move the mask with the layer. Back in the CameraTracker effect, we can now select that
comp we just created as the Matte Layer, setting the Matte Source to Matte Alpha. Now the CameraTracker
will not track any features within that mask shape we created.
We can now move on to tracking our features. To preview the features before tracking, I’ll
set the Analysis Range to only track the first frame. After clicking the Track Features button,
we can see the features that will be tracked. I’ll open up the Tracking settings, and
I’ll increase the Number of Features to 400. This will take longer to track, but it
will give the CameraTracker a lot more information to work with to get a better result. In most
cases you should use no less than 100 features. Hit Track Features again. We don’t want
any features being placed on areas with very low detail like the sky, so if you see any
features there, try to increase the Detection Threshold. I also want these features to be
spread out a bit more, so I’ll increase the Feature Separation slightly. The camera
is translating (moving) and rotating in this video, so I’ll leave Track Validation as
Free Camera. If your camera was placed on a tripod and is only rotating, like in this
second clip, set this to Rotating Camera instead. With these Tracking settings in place, we
can go ahead and set the Analysis Range back to the full Source Clip Range, and click Track
Features. CameraTracker will track through the shot forwards and backwards.
Before solving the 3D camera, it’s a good idea to review our tracks and remove any that
are poor quality. I’ll start by simply scrubbing the timeline and looking at the trackers.
If any of them seem to move or slide around in a weird way, you can select them and delete
them. Some of these tracks for example aren’t locked onto anything and are sliding across
the building, so I’ll go ahead and delete them. We can also automatically remove certain
trackers based on some threshold values. Open up the Refine settings. Since we’re currently
in the tracking phase, we are only concerned with the Track controls for now. Most of these
settings are only displaying data for us to reference. We shouldn’t change the values
as doing so will have no effect.
The main value I’m interested in is Track-Lifetime-Min. Select it by double clicking, then open up
the Value Graph. This value shows the shortest lifetime out of all the trackers on the current
frame. Compare these values to the next graph, Track-Lifetime-Avg. This shows the lifetime
of all the trackers on the current frame averaged together. The average is much higher than
the minimum as expected. Towards the bottom of the list are four threshold settings. These
settings we can change, and they control which tracks we want to “reject” and delete.
As I increase the Threshold for Track-Lifetime-Min, you can see some trackers turning red, meaning
they don’t fit within the threshold. A value of 12 means that any tracks that don’t last
for at least 12 frames will be rejected and removed. Depending on your shot, tracks with
very short lifetimes are often good candidates to be removed. Click Delete Rejected to remove
these trackers.
Once you are happy with your tracks, we can move on to solving the camera. In the Solve
controls, enter the correct information for the Focal Length of the lens you used and
the Film Back or Sensor Size if you have it. If the lens is zooming in or out in your video,
choose one of the Varying options. If your camera was placed on a tripod and is only
rotating, choose Rotation Only for Camera Motion. Click the Solve Camera button, and
CameraTracker will calculate a 3D camera based on our tracking points. Scroll down to the
Refine section again and look at the Solve-Overall Error. This number gives us an idea of how
accurate CameraTracker thinks the solution is. An error between 1 and 2 seems to be pretty
normal, but let’s see if we can reduce it.
Now we can look at the Solve Refinement options to remove any problematic trackers and get
an even better result. Solve-Per Frame Error-Avg shows the overall quality of the solve over
time, based on the average error of all the tracks on the current frame. Compare this
graph to the Solve-Per-Frame Error-Max, which shows the highest error value of any tracker
on the current frame. Here we can see some pretty large spikes. I’ll delete these high
error tracks automatically with the threshold setting. After hitting Delete Rejected, our
overall error is lower. Nice! The next value I’ll take a look at is the Solve-Per Track
Error-Max. I’ll lower the threshold to get rid of any low quality trackers with a high
track error. You may still see some red trackers. These are called Unsolved, and don’t have
a valid 3D point in the solution. Some of them don’t seem to follow the proper camera
movement, so we can delete them by hitting the Delete Unsolved button. There! Our solve
error is much lower than before, so our resulting 3D scene should stick onto the footage nicely.
Now that we’re happy with our solve, we can go ahead and create the 3D scene to work
with in HitFilm. Click the Create Scene button. This adds in a new 3D Camera layer parented
to a new 3D Point layer. If you scrub the timeline, you can see the camera moving. HitFilm’s
Floor Plane doesn’t seem to be aligned correctly. To fix this, select the CameraTracker effect
again. Click and drag a box selection over some tracking points that are on the ground
in your footage, then in the Tracker menu, choose Ground Plane, Set to Selected. This
will match the tracked scene’s ground to the average position and orientation of those
points. You can also scrub the timeline, select more points, and click Set to Selected to
add more tracking points to the averaged ground plane. Now HitFilm’s floor plane seems to
stick right onto the ground in our video footage. Fixing the ground place like this will not
work if your camera was only rotating. If you click Toggle Render Mode on the CameraTracker
effect and create a new Camera layer, you can see a Point Cloud representing the tracking
points. As you can see, there is no actual depth information when the camera is only
rotating. Going to one of the other 3D views, such as Top View, our 3D scene was created
far away from HitFilm’s world center. You can relocate the scene by moving the point
that the camera is parented to. If your scene also appears to be upside down, you can rotate
this point to realign it. Adding in 3D objects will be easier now that the 3D scene is oriented
correctly.
Let’s add some 3D objects to utilize our tracked scene. With the CameraTracker effect
selected, you can create 3D Points and Planes from the tracking points. Select three or
more points and create a Plane. This 3D Plane should appear to stick to the surface it was
created on. If the angle seems strange you can adjust the rotation manually or try selecting
some different points to start with. Now we can bring in another layer, such as a logo
graphic. Make the layer 3D, parent it to the plane, and reset it’s transforms. With a
few position, scale, and rotation adjustments, we now have the start of a neon sign or billboard
effect! This isn’t only useful for signs and logos. Use a tracked plane to add some
texture to a surface such as grunge or cracks in your concrete. Make sure to use a feathered
mask to hide the edges of the texture. How about some floating text? Select a tracking
point close to where you want your text to be, and create a Point. Now make your text
layer 3D, parent it to the Point, reset it’s position, then reposition it. Maybe you want
a 3D model sitting in your scene. The ground plane we set earlier is not accurate towards
this side of the road, so I’ll select some trackers in that location and add a new plane.
If you want your model to stick to a certain surface, make sure it is actually sitting
on that 3D plane. The next step would be to composite these elements into the scene with
some proper lighting, shadows, and reflections to increase the realism. With some more time
and effort, you can take your video from looking like this, to looking like this!
Foundry’s Camera Tracker in HitFilm Pro is incredibly useful for a wide variety of
effects. Feel free to leave a comment if you have any questions or if you learned something
new. Subscribe for more tutorials like this, and be sure to give this video a like. See
you next time!