Hello everyone, my name is Farshid. Today I’ll be presenting AuraRing, our work on
Precise electromagnetic finger tracking for mobile device input. First, I want to thank
my co-primary author on the paper, Eric Whitmire and my advisor Shwetak Patel.
New computing platforms, such as head-mounted displays for virtual and augmented reality
have the potential to change the way we work, play, and communicate.
More and more, our computing surfaces are not limited to the smartphone screen in our
hands but are placed ubiquitously throughout the environment.
Wearables are great solution for these input devices as they are continuously available,
but most are focused on detecting events, like taps, flicks or other gesture. These
are useful but we need to complement this gestures with precise continuous tracking
for pointing.
To enable precise tracking, we built AuraRing. It’s a high precision short-range electromagnetic
tracking technique that goes beyond discrete events and tracks the pose of the ring with
respect to the wristband in real time. So we can use this to reconstruct hand pose
or we can build continuous input applications like drawing on top of it.
Now let’s talk about how this works, AuraRing consists of a wristband, something that could
be integrated into a smartwatch or other wearable, and a ring, much like today’s smart rings.
Our goal here is to enable interaction in a manner that is subtle, portable, precise
and does not require user calibration.
Before we get into the interactions that we enable, I want to review how electromagnetic
tracking works. When we pass current through a coil, it generates
a magnetic field in a direction normal to the plane of the coil. When you put another
coil, here we call it the receiver coil, within this field, the voltage in the receiver coil
is related to the orientation with respect to the field. If the receiver coil is aligned
with the field, you measure the maximum voltage. If the receiver coil is orthogonal to the
field, you measure nothing in the receiver coil.
Taking a closer look at the hardware, on the left you can see the ring. We literally wrapped
800 turns of thin wire around a 3D printed frame and put a small PCB on top to generate
the signal. The ring is actually a very small battery-powered coil that emits a single magnetic
field around the hand and uses just a couple of milliwatts, so enough for about a day’s
worth of usage.
On the wristband, you can see the three sensors here, they are low-profile sensors that contain
three axes each and measure the field at three different locations. Since we know the geometry
of the three sensors, we can estimate the pose of the ring. We also have two controller
boards that handle signal processing and sampling. For more details look at the paper
Here we show the raw data that the wristband sends back to the PC. As you see it’s quite
sensitive to the finger movement. Now how can we relate these measurements to
the ring pose? To do that, we start with a magnetic dipole
model that calculates the sensor measurement as a function of the ring pose for each sensor
location.
Then, We use a non-linear optimizer to solve the best position and orientation given a
set of the sensor values.
However, the sensors and amplitude demodulation pipeline do not represent a perfect measurement
of the magnetic field due to sensor parameters including noise, gain, bias and etc.
That’s why we need a calibration process!
we learn these one-time factory calibration parameters empirically by collecting and using
data from an optical motion capture system . We placed IR retroreflective markers on
the ring and wristband to get ground truth and we use that to learn the forward model.
To evaluate the calibration, we look at the correlation between what the sensor measured
and what the forward model estimates. These should be perfectly correlated with a perfect
calibration. Without any calibration as you see the measured
sensor value does not correlate with simulated sensor values. After calibration, data measured
by AuraRing match the synthetic data.
To evaluate our system, we recruited 14 participants and asked them to wave their hand and finger
for 10 minutes. In this evaluation, we wanted to understand how well the calibrated model
track the ring.
And here are the results. You can see a representative few seconds of 3D positional tracking for
two of the participants. The takeaway here is that AuraRing tracks the pose with mm resolution
accuracy. One thing to focus here is that even for the user on the right which has relatively
higher error, the relative motion still tracks the ground truth motion.
For more details please look at the paper Now I like to show you some of the interactions
AuraRing enables. As we saw earlier, a user can precisely point while keeping their arm
rested at their side. We can leverage the pose estimate to fit a
kinematic model of the hand. The does require knowledge of two bone length
which can be learned automatically. AuraRing can also enable interactions with
virtual objects. One interesting aspect about AuraRing is that
due to its high bandwidth, we can measure events such as taps or flicks which are immediately
distinguishable due to their broadband spectrum and higher energy
By combing AuraRing with Arm or wrist tracking solutions, pointing can be done in absolute
coordinate. Here we show a drawing example using wrist-only motion from the HTC vive
tracker on the bottom and wrist motion + AuraRing on the top. You can see that AuraRing results
in a significantly more useful signal. With that I want to thank you all for listening
and again thank my co-authors. And would be happy to take any question.