Hi, everyone. I’m Yuzhou Zhuang from Tsinghua University.
Here I present ReflecTrack, a novel method that enables 3D acoustic position tracking
using commodity dual-microphone smartphones. This work is a collaboration between Tsinghua
University and the University of Washington. 3D tracking is not easily accessible in daily
life. On one hand, interaction on smartphones has
been limited to 2D space for decades, especially dominated by the 2D touchscreen.
On the other hand, modern VR systems support large-scale fine-grained 3D tracking for immersive
and interactive VR experience, but they rely heavily on external sensors or dedicated hardware
which is not accessible in daily life. In recent years, audio hardware such as speakers
and microphones have been widely available and increasingly popular on commodity mobile
devices. These acoustic sensors could potentially enable
convenient and accessible 3D tracking on smartphones. Previous work has explored many acoustic tracking
solutions on mobile devices, including Doppler effects, correlation, phase and FMCW.
Among these systems, FMCW has become the state-of-the-art for its sub-millimeter level accuracy.
However, these systems either do not support 3D position tracking or require dedicated
hardware. The fundamental challenge for 3D tracking
with only a smartphone is that triangulation requires at least three distances for 3D positioning,
while most commodity smartphones have at most two accessible speakers or microphones.
Previous work also explored the use of acoustic echoes to facilitate sound source localization,
but is limited to angle estimation with simulated data.
This paper, instead, proposes and implements a novel 3D acoustic position tracking method
leveraging everyday reflective surfaces and the built-in dual microphones that exist in
almost all modern smartphones. In order to track the 3D position of an external
sound source with only two microphones, we introduce a reflective surface near the smartphone,
which effectively creates two mirror microphones, allowing us to obtain four distances using
ultrasonic ranging techniques and perform triangulation to infer the 3D position.
The key to leveraging echoes is a general reflection model that defines mirror microphones,
and the echo-aware FMCW approach that simultaneously estimates the direct and
reflective distances from a single microphone.
To achieve this, echo-aware FMCW improves the traditional FMCW approach with the triangular
chirp signal and the CFAR peak detection process. If we take velocity into account, we can derive
the range equation with an error term caused by the Doppler effect.
In triangular chirp based FMCW, we integrate the two reversed FFTs to cancel the motion
effect as well as enhance the signal-to-noise ratio. Echo interaction can also result in ghost
peaks and missed peaks in the FFT profile. In order to robustly identify the two peaks,
we adopt the Constant False Alarm Rate peak detection algorithm, which provides a consistent
process for detecting multiple targets with adaptive thresholding.
The FFT spectrogram shows that echo-aware FMCW accurately and robustly finds the direct
and reflective paths. Once we obtain two direct distances and two
reflective distances, we can perform triangulation to infer the 3D position of the sound source
relative to the smartphone. ReflecTrack achieves a 3D tracking accuracy
of 28.4mm in a cubic space with 60cm side length and 22.1mm with 30cm side length.
Reflective surfaces can be found in everyday objects such as walls, tabletops, partition
panels, and packaging boxes, as well as fabricated from raw materials such as acrylic, wood,
glass, and cardboard. We conducted extensive evaluation study to
investigate the effect of both internal and external factors.
First, different reflective surface materials and placements show no significant difference
in the 3D tracking error, proving that ReflecTrack is a general tracking scheme.
Second, it is important to select an appropriate surface-to-microphone distance.
Our evaluation shows that 3D tracking error remains low between 14cm and 24cm.
Comparison with traditional FMCW shows that triangular chirp based FMCW is significantly
more accurate and robust to motion effect, with 17.5% decrease in the median 3D tracking
error. ReflecTrack is also resilient to environmental
noise. Finally, we conducted a user study with 10
participants and collected 40 sessions of data.
From the heatmaps of the 3D tracking error averaged by all participants, we found that
ReflecTrack generally performs better in the central area than on the edges.
Further ANOVA analysis indicates that the 1D distance error for the direct path is significantly
lower than for the reflective path, and the 3D tracking error in the high-accuracy region
is significantly lower than in the entire tracking space.
Here, we show that the 3D tracking error remains low and stable when the motion speed does
not exceed 3m/s, and the earbud and the smartwatch have similar performance as the sound source.
Finally, we present some demonstrative applications using everyday surfaces and objects, including
viewpoint manipulation of a 3D object, fine-grained in-air gesture control of a video player,
and motion tracking in mobile VR. To summarize, we presented ReflecTrack as
a general scheme for enabling relative 3D position tracking using commodity dual-microphone
smartphones. We proposed the echo-aware FMCW approach to implement the system.
We presented evaluation results that showed ReflecTrack achieves high tracking accuracy
in a large tracking space. We demonstrated that ReflecTrack is easily
accessible through several demonstrative applications using everyday surfaces and objects.
Thanks for watching! Please look at our paper for more details.