Hello Everyone, I am Hem Regmi. I am the PhD student at the university of south carolina.
Today, I would like to present our paper
SquiggleMilli, approximating SAR imaging on mobile millimeter wave devices
Before we talk about the details of the paper, lets talk
about the mmwave image because of the mmwave image
we can detect the defective pipes, wires etc. beyound the wall without destroying the wall
we can even defective package without opening the package itself
All of us might have seen this in the north american airport
where they use mmwave technology to detect hidden object inside
all of these are possible because of the small wavelength property of millimeter
our question is can we bring this function to commodity 5g smart phones
that have mmwave technology
this is where our paper is focused on, where we like to think if we can pull out our smart
phone , squiggle over some object of interest and it somehow can show what object is inside
To explain more precisely, lets compare our system with the vision based camera. We can
see if the object is not occluded then vision based and squigglemilli produces the similar shape
however if the object is hidden inside the cloth, then vision camera fails to reveal
what object it is, while the squigglemilli can easily produce the 2d shape and we can guess
what kind of object it is, as we can see in the right bottom figure
We need to solve few challneges before we achieve that
The first one is hand-held imaging, where we have non-linear motion,
non-uniform sampling and multiple objects in the scene
Second challenge is the specularity of mmwave and weak reflectivity of the object, where the
transmitted signal strikes the object at certain angle and doesn't reflect back to antenna.
And we will be missing of lots of surfaces of the object in mmwave image
This is where we design our system to solve all these challenges. In the left side, we
first apply the non-linear motion compensation to compensate for non-linear motion and then we apply
the compressed sensing technique to recover the missing samples,
and after that we segment each object if there are multiple object present in the scene
After that we generate the mmwave and supply that to the trained generator and discriminator where
it produces the high quality 2d shape. It supplies the 2d shape to the quantifier and classifier
to predict 3d features and object class. So in total, we start with reflected signals
and squiggle aperture locations and finally generate 2d object shape , 3d features and class.
to solve hand-held imaging challenge, we use multiple receiver antennas to find
nearest grid locations to compensate for non-linear motion. We use standard
compressed sensing technique to recover missing samples after motion compensation is applied
As you can see in the figure, before improvement we can not even recognize
what kind of object it is , while after the improvement we can guess its scissors
We use density based clustering method to segment each object
if we have multiple object in the aperture
Another challenge to solve is specularity of mmwave and weak reflectivity of the object.
this happens because the object surface acts like mirror and the transmitted
signal bounce off an angle and will not comback to receiver
as we seen in the object B. And the top figures shows different objects
with corresponding mmwave image where we are missing most of the surfaces
To solve this challenge, we use machine learning technique where we pass the low resolution mmwave
image and corresponding ground truth shape to the conditional generative adversarial networks
which learns from thousands of samples in training to produce the 2d shape
as we see in the right side, improving shape
after the training done, we only need generator that is trained and low resolution mmwave 3d image
to produce the high quality 2d shape, which is human perceptible
we use generator and discriminator to improved 2d shape and classifier
and quantifier defined on the top of that, to predict the 3d features and object class
to implement this system,
we first the ask volunteers to squiggle the phone to collect the pose data.
We then collect the ideal mmwave data from the TI mmwave radar.
we now apply the pose on these ideal mmwave data to obtain the squiggle mmwave data
at the same time, we collect the ground truth 2d shapes by using the co-located AR camera
to evaluate our system, we use multiple partially and fully occluded objects
as we see in the partially occluded pistol, the
traditional SAR technique wil not be able to produce high quality image
while squigglemilli is able to produce high quality shape which is human perceptible
this holds true even for the fully occluded pistol as we see in the middle row figure
we also tested for scissors and this still holds and we can see the recovered shape
finally to summarize, squigglemilli brings the high resolution thorough obstruction
imaging into cheap, ubiquitous mobile devices
squigglemilli improves the shape to make human perceptible using machine learning techniques
please feel free to check out our paper for more design details and results. thank you