we introduce radarnet an efficient swipe
recognition technique using a
miniaturized radar sensor called soli
our novel algorithm recognizes swipe
gestures using unsegmented time series
radar signals with minimal computational
resources
the sole radar system was integrated
into the top bezel of a mobile phone
demonstrating its small footprint is fit
for consumer products
radarnet our convolutional neural
network is designed to detect gestures
with radar signals
the radar net consists of a signal
processing module a frame model
a temporal model and a gesture debouncer
the algorithm outputs three predictions
portrait prediction
landscape prediction and omni swipe
prediction
these predictions detect left right
swipes up down swipe
and omniswipe which is a swipe in any
direction
the radar signal processing applies a
series of ffts
to raw radar samples calculating a range
doppler complex map
radar net uses 12 frames of a range
doppler complex
from three receiver channels shown here
as an input
the input is processed by the frame
model first the frame model has
convolution
pooling and relu layers utilizing
residual blocks and bottleneck blocks
through the model the input is
summarized into a one-dimensional vector
with 32 values
the temporal model concatenates the
summary from the current frame
with the summaries from the last 11
frames and passes them to an lstm layer
the output from the lstm layer is then
passed to three dense layers with
softmax
which output the three sets of class
probabilities
radarnet outputs class probabilities for
a given segment of 12 frames
however in practice an algorithm must
recognize gestures from a continuous
data stream
to realize this we added a gesture
de-bouncer on top of the radar net
the gesture d bouncer generates a
gesture detection event when the gesture
probability becomes higher than a
predefined detection threshold
then it suppresses the gesture detection
events until the probability becomes
lower than the debounce threshold
and the debounce timer expires to avoid
generating multiple gesture detection
events for one gesture
we collected a large-scale data set in
our organization
to train and test radarnet the dataset
has 558 000 gestures
from 7647 participants at nine different
google offices
as well as 285 hours of negative
recordings where people performed
motions similar to gestures in many
different contexts
we split the recordings into training
development and test sets
the data split into these sets was done
on a user basis
meaning data from the same participant
was not split into multiple data sets
we evaluated the performance of radar
net in many aspects
including computational efficiency
segmented classification
and unsegmented detection we assessed
the computational efficiency of the
radar net
based on its model size and inference
time the model sizes were measured using
file sizes of the model in the tf lite
format
the inference times were measured using
a tf light profiler
we used two models as bass lines the
first was an end-to-end model
is a model that recognizes gestures with
the solely sensor
the second was mobilenet a general image
recognition model designed for mobile
devices
the table shows that radar net is
significantly more efficient than the
baselines
in the segmentation classification task
recordings in the test set were
segmented into samples with ground truth
labels
similar to the training process we
evaluated the performance of algorithms
based on how well the model could
predict the ground truth labels
the accuracy values are higher than 0.99
showing that radar net has high
performance in the segmented
classification task
furthermore we evaluated radarnet in an
unsegmented detection task
in this task recordings in the test set
were used without segmentation
the model made predictions for every
frame using the continuous data as input
the graph shows that the detection rates
are higher than 0.9
while false positives per hour values
are lower than 1
indicating robust gesture detections in
the practical setting
in summary radar net recognizes swipes
using privacy preserving radar signals
our experimental results demonstrated
that radar net is efficient enough
to be continuously executed on devices
with limited computational resources
while also providing reliable gesture
recognition performance
with practical real-world contexts the
swipe gestures recognized by our system
opens up new opportunities for further
research and product integration
using radar-based sensing in the context
of ambient computing