Hello everyone.
In this video I'll introduce our work:
3D point cloud generation with mm-wave radar.
A wide range of sensing modalities
are already available on vehicles,
such as cameras, lidar, and radar.
Visual sensors like cameras and lidar
are vulnerable to adverse weather conditions
such as rain, snow, and fog.
In contrast, radar can penetrate small water
particles and can make autonomous driving
more robust against bad weather.
However, vehicle radar has two main drawbacks.
First, the radar has low spatial resolution
due to its small form factor.
While visual sensors reveal fine-grained
shapes of targets, radar only generates
intensity maps with strong reflection peaks.
Second, vehicle radar typically emits mm-wave signals,
which are specularly reflected by most objects
and may not be captured by the Radar.
These figures show the aligned camera images
and radar intensity maps of different vehicles.
The radar can only see parts of the vehicles
with strong reflections towards the radar.
Recent innovations in mm-wave sensing
noncoherently fuses the measurements
along the Radar’s moving trajectory.
While to some extent alleviating the
specular reflection problem by illuminating
the target from diverse locations,
their imaging resolution is still
limited by the physical size of the
antenna array and cannot be fundamentally
improved through spatial sampling.
To solve this problem, we propose MilliPoint,
which enables synthetic aperture radar
imaging with low-end vehicle radars.
MilliPoint coherently combines measurements
of the radar, to generate dense
and high-resolution 3D point clouds.
The first challenge is that SAR requires
accurate tracking of the radar.
The figures below show two cases
where the radar moves along the
cross-range direction at a constant speed
and an accelerating speed.
Without the knowledge of the
radar's location, the image of the
object can be highly distorted.
To solve this challenge, we propose
a self cross-range tracking method.
We let the radar estimate its instantaneous speed.
We exploit an observation that different
transmit-receive antenna pairs of the
moving radar experience similar channel
responses with a time delay.
An intuitive example is shown in the left figure,
the radar has equally spaced 4 antennas.
As the radar moves, the 1st antenna pair at time
t1 coincides with the 2nd antenna pair at time t0.
So their measurements become highly
correlated with a time delay of t1-t0.
Meanwhile, the radar moves by one
antenna spacing, so the speed
of the radar can be calculated.
In general, for any two antenna pairs,
the cross-range movement is the average
of the spacing between the transmit antennas
and that between the receive antennas.
The elapsed time is the delay time of the
measurements of the two antenna pairs
with the highest correlation values.
The second challenge is that SAR
requires focusing on the center of the target.
The figures below show the scenario
where a metal surface is 3m away
and has an incident angle of 135 degrees
with the cross-range direction.
If the SAR focuses on the broadside direction,
then the target disappears, as in the left figure.
Only when the SAR focuses on the direction
of 45 degrees, the target is imaged,
as in the right figure.
The root cause of this phenomenon is the
mismatch of the physical aperture of the radar
and the effective apertures of the targets.
The physical aperture represents
the physical trajectory of the radar.
The effective aperture of a target
is the segment of the trajectory where
the target's reflection is received.
As shown in the figure below, due to
specular reflection, the effective aperture
of the metal surface is the short
red segment, while that of the wall
is blue segment and the same as
the physical aperture of the radar.
In the paper, we prove that the focusing center
can be represented by the parameters
l and theta, which can be identified
in the spatial frequency spectrum.
On this basis, we propose the
multi-focusing algorithm to overcome
the specular reflection problem and
image all objects in the environment.
The final challenge is that SAR
only generates 2D intensity maps.
The figures below show an indoor
scenario, where images of objects at
different heights overlap with each other.
To solve this challenge, we exploit
multiple antenna pairs along the
vertical direction. For each antenna pair,
we generate one SAR image and
find pixels with high intensity.
In the paper, we prove that the
complex pixel values still encode
the elevation information.
So we combine co-registered pixels
of all SAR images to form a virtual
pixel array and estimate
the elevation of the pixels.
The right figures show an indoor scenario
captured by a stereo camera and the
radar point cloud generated by MilliPoint.
We conduct experiments using a
TI cascade radar with 6 transmit antennas
and 8 receive antennas.
A ZED stereo camera is used
to capture the ground truth.
We compare MilliPoint with
non-coherent imaging and
the imaging with a static radar.
In different scenarios with different
objects such as cars, pedestrians,
bikes, and trash bins,
MilliPoint generates the densest
point clouds and the points are
accurately aligned with the objects
and reflect their shapes.
In conclusion, in this paper,
we propose MilliPoint, which is
the first step to enable high-resolution
high-density 3D point cloud generation
on low-end vehicle Radar based on SAR.
We envision MilliPoint as a
new type of sensor fusion modality.
Thank you and I'd like
to take any questions.