Labeling point clouds is one of the most tedious tasks
in deep learning workflows for LiDAR.
So MATLAB is providing a LiDAR Labeler app in LiDAR toolbox
which alleviates some of the most common pain points
of point cloud labeling.
In this video, I'll show you how to label your point cloud
data using LiDAR Labeler app for applications
like autonomous driving, robotics,
and augmented reality.
We'll first see how to ground truth label point
cloud for object detection.
And then, we'll go over how to automate the labeling process.
Let's start by opening the app from App menu.
We can find the LiDAR Labeler app
under Image Processing and Computer Vision section.
Alternatively, we can open the app from command line
by typing LiDAR Labeler.
We'll first load the point cloud into the app
by clicking Add Point Clouds.
You can also import labels from files, from Workspace,
or you can open a session you have already saved before.
We can load raw point clouds in different file formats,
like PCD, VLY, PCAP, LAS, and LAZ.
Here, we are loading sequential point clouds
off a highway driving scenario in PCD format.
Once the point cloud sequence is loaded, as a best practice,
we can play the entire sequence to get
an overview about the scenario and objects
in the point cloud sequence.
There are some other features that
will help you to get a better understanding about the point
cloud before labeling.
For example, you can hide the ground points
using Range-based Floodfill or Fit Ground Plane methods.
You can also use the Snap to Cluster
feature to apply different types of clustering on the point
cloud data.
Let's run the sequence again now.
Now that we know the objects we need to label in the sequence,
we can define them by clicking Label from the ROI Label
tab in the left pane.
Let's give the label name as Car and treat cuboid
over the cars in the scene.
We can select this Shrink to Fit option
so that the cuboids fit properly over the object clusters.
Now, let's label the car in the sequence
by clicking on the car cluster in the point cloud.
We can use Projected View option to see
different views of the selected objects
and then manually fit bounding box over the object.
The ROI feature in the app enables
you to define the ROI and label objects in the ROI.
You can change the size of points in the point cloud
to visualize the clusters.
Another useful feature in the app is Camera View option.
You can save and reuse custom camera views of the point cloud
data.
Now that we have seen how to label objects in the point
cloud, let's talk a little bit about automating
the labeling process.
For that, we'll go to the Label tab
and select the Automation Algorithm.
In this case, let's select the Point Cloud Temporal
Interpolator.
This algorithm estimate cuboids in the intermediate frames
using interpolation between the cuboid arrows
in the key frames.
We'll then click the Automate button
and manually label some key frames.
Here, we have selected five random instances as key frames.
After this, we'll select the Automate tab and click Run.
Now you can see that all the intermediate frames
are labeled automatically.
You can go through the label frames
and can manually edit any of the frames as per needed.
Finally, you can accept the automation
and export the ground truth label point
cloud to MATLAB workspace or to a file.
Alternatively, if you have a custom automation algorithm,
you can use them by selecting Add Algorithm option.
We can then import the algorithm.
We are following this workflow example
to demonstrate how to use a pre-trained object detection
network to automate your LiDAR labeling process.
Here, we are using a pre-trained point pillar object detection
network that can fit oriented bounding boxes around vehicles
in the point cloud data.
Once you have imported the automation algorithm,
you can find it under Select Algorithm panel.
Now, click the Automate button, followed by the Run button.
You can now see the bounding boxes
around the detected objects.
If the detection seems fine, we can accept the automation
and export the ground truth label point
cloud to MATLAB workspace or to a file as we've done before.
Here, I am exporting the labels to MATLAB workspace.
A quick look at the data source will
show us the label properties, like source name,
signal name, signal types, et cetera.
We can further use the label data
set to train an object detection network on your LiDAR point
cloud data.
Follow the links below to learn more.
If you have any questions or comments, please let us know.