Hi, everyone.
I'm Jian gong from central south university.
Today I'm going to share our latest
research Robust Inertial Motion
Tracking through Deep Sensor Fusion,
across Smart Earbuds and Smartphone.
Inertial sensors.
IMUs, including gyroscope,
accelerometer, and magneometer,
have been a standard module in modern
mobile devices and play a fundamental role
in many ubiquitous computing applications.
Example use cases include tracking users'
trajectory in VR gaming,
navigating a user in GPS
denied indoor navigation,
serving as the low power fitness tracker,
complementing GPS in last-mile scenarios.
The underlying
algorithms, often referred
to as inertial odometry,
or dead reckoning,
aim to estimate the moving
direction and distance of the user
relative to the earth coordinates.
The global reference frame.
Unfortunately,
track human mobility with on-body IMUs, the
sense readings are intrinsically noisy.
The motion noise are generated
unconsciously and inevitably,
either by limb swinging,
shaking during normal walking,
or by, ordinary maneuvers,
such as phone rotation,
are moving into pocket.
Such motion artifacts are
indistinguishable from walking-induced IMU dynamics.
And the resulting error will be
further magnified by gyroscope
or accelerometer integration,
resulting in exponential error
propagation over a walking trajectory.
In other words,
it is the regular motion artifact itself
that plagues ,
inertial motion tracking in practice.
In this paper,
the proposed deep inertial tracking
DeepIT, a deep learning based
sensor fusion framework to enable
truly robust inertial motion
tracking in the wild.
Our key insight is simple: to
combat the motion noise,
we need to incorporate additional
IMU sensors whose measurements
are not likely to be contaminated,
at least not simultaneously with the
smartphone's IMU measurements.
We realize such sensing
diversity using smart earbuds,
which are emerging as a new
generation of wearable devices.
In many of the desired use
case of inertial tracking,
such as indoor navigation
and outdoor run TRACKER.
The earbuds are accompanied
by the user's smartphone.
Smart ear are far less susceptible
to noise pollution
than handheld smartphones.
Because head rotation occurs much
less frequently and at much smaller
scale compared with the body limbs.
Our deep ID model segments
the input IMU sequences into small windows,
and outputs the polar coordinate
representation of the user's
location. To effectively harness
complementary capability of
smartphone and earbuds,
we design a reliability network,
which gauges the confidence of sensing
from each IMU, which IMU is suffering
less from motion artifacts
momentarily.
The reliability network then forwards
its output to an attention module,
which fuses the two IMUs's readings
based on their reliability.
Observing that the IMU data is a continuous
stream of samples and motion artifacts.
despite burstiness,
do exhibit short-term correlation.
Therefore,
we cast the reliability network design
as a sequence learning problem and
construct it based on the LSTM model.
Even with sensor fusion,
the angular and radial components of
localization are still susceptible
to cumulative error caused by the
summation of each windows's offset.
Whereas the drift
of radial components cannot be
compensated by additional measurement.
The angular counterpart can be
calibrated by magnetometer
reading,
which is not susceptible
to accumulative error.
The noise level of both IMUs,
represented by reliability vector,
is adopted as a metric to determine
how much the magnetometer reading of
the phone's IMU can be relied on.
Based on the value of reliability vector,
the magnetometer reading is then fused with
angular component determined by the
neural network for more accurate
angular offset estimation.
There is no public dataset for
inertial tracking using both
smartphone and earbuds simultaneously.
Therefore,
we collect our own dataset with pedestrian
carrying Android phones and wearing smart
earbuds.
We use the eSense, an
experimental smart earbuds device
developed by Nokia,
which can stream gyroscope and
accelerometer readings through
BLE to an associated smartphone.
Our dataset covers a comprehensive
set of real world usage settings.
To examine the performance of DeepIT
and benchmark solutions under
different levels of motor noise,
the data set incorporates 4 types of walking
patterns:
normal walking (NW),
walking while swimming phone occasionally (RP),
walking while swinging phones and moving heads alternatively (RPE),
walking while moving heads occasionally (HM).
In NW scenario,
we can see that the reliability of the
phone is always higher than the earbud.
This is because the phone
endures clearer periodic walking patterns than the earbud,
so its translation estimation is more accurate.
In RP scenario,
the earbud's reliability is
always higher than the phone.
This matches our expectation because
the phone endures more motion
noise in this scenario.
In RPE scenario, the two sensors' reliability level change
alternatively.
Comparing with the original IMU data
within the same time frame in Fig. 6,,
we find that the lower reliability period
precisely corresponds to the moments when
the sensor endures motion noise.
This indicates our liability
aware attention scheme
faithfully tracks the motion noise intensity
and adjusts the fusion weights accurately.
Due to time limitation,
we only reduce the high-level
ideas of our research.
For more details,
please refer to our paper,
Robust Inertial Motion
Tracking through Deep Sensor Fusion
across Smart Earbuds and Smartphone.