Hello everyone, I will present our paper titled MechanoBeat:
Monitoring Interactions with Everyday Objects using 3D Printed Harmonic Oscillators and
Ultra-Wideband Radar.
I am Farhan and this work was done in collaboration with my colleagues Julian, Dave, Huaishu,
and Tauhidur.
Knowing how and when people interact with their surroundings is crucial for constructing
dynamic and intelligent environments.
These interactions are important for creating life logs, smarter homes, smarter workplaces,
and potentially facilitate ambient assisted living.
Current solutions are often costly and may require significant maintenance burden or
suffer from privacy concerns, line of sight requirements, and restricted placement options.
Our goal was to design simple, low cost, easy-to-install tags that do not require any batteries and
can generate unique oscillation patterns upon interactions
which can be monitored without direct line-of-sight.
We also required a reset mechanism to stop tag oscillation in order to differentiate
between consecutive interactions.
The proposed tags can be classified into two categories: Stationary and Mobile.
All of our tags can be printed on hobbyist grade 3D printers using cheap plastic filaments
and require no power and minimal maintenance.
MechanoBeat uses PulseON 440 Ultra-WideBand radar operating at 3.1 to 4.8 GHz frequency.
Due to its narrow pulse duration, Ultra-WideBand radars have very high range resolution which is appropriate
for measuring subtle motions like human breathing and heartbeats.
This use case gave us the idea to replicate a heartbeat using harmonic oscillators.
The radar unit, along with an absorber material behind the antenna, a Raspberry Pi, and a
hard disk drive are placed in a 3D printed T shaped box.
Radar units send out radio pulses that are reflected off of objects in the surrounding
environment and received by the receiver antenna.
By keeping track of how long it took for the pulse to return, the radar system can get
an idea of how far away an object is.
In the radargram on the right, the Y-axis represents time where each line is a pulse.
The X-axis represents the distance from the radar while the color represents the magnitude
of the reflected pulse.
The main stationary tag uses a pendulum to generate periodic oscillation.
There are three main parts of the tag.
The first is a 3D printed arm having a metal ball on one side.
Second is the stand which attaches to the application and acts as a pivot point for
the arm.
Finally, we have the reset arm which stops the motion when an application is not in use
by pushing the arm back.
By adjusting the arm lengths and leveraging both single and double ball options we can
create distinguishable tags in a variety of oscillation frequencies as shown in this table.
We instrumented a kitchen environment with various types of stationary tags and mounted
2 radar boxes on adjacent walls.
In each experiment, the tagged appliances were interacted 10 times each lasting around
10 seconds.
We recorded the start and end time, tag location, and interaction duration of each round for
ground truth data.
We used a three-second time window with a one-second shift to convert our continuous
time-series data to discrete instances.
In the first part of our pipeline, we adopted a 1D CNN architecture that consists a total
of six 1D convolutional layers split into two sections by a mean pool layer.
At this point, the model outputs an embedding of length 128.
The embeddings for each range bin in a given time window are then combined and passed through
a fully connected network called multi-range bin model that outputs the final tag class.
The results show that MechanoBeat is able to accurately differentiate various tags both
in line of sight and through the wall scenario with an F1 score of 0.9.
Our system could also detect and identify simultaneous interactions with 2 tags with
reasonable accuracy
Our mobile tag design is inspired from tourbillons used in mechanical watches.
A basic tourbillon has a core revolving structure including a balance wheel, a pallet fork,
an escape wheel, and a hairspring.
The balance wheel is the 'beating heart' of the tourbillon which oscillates when the lid
is opened or closed.
The mobile pill bottle tag is handheld by a participant while walking towards four different
chairs one after another located at different positions within an indoor space.
While seated in each chair, the mobile tag is interacted with for 10 seconds by opening
the lid.
A localization algorithm tracks the trajectory of the moving person as shown on the left
and our classification pipeline was able to successfully detect the active instances of
the pill bottle tag with reasonable inference probability as shown on the right.
In future, we intend to develop and test algorithms for more complex environment with multiple
individuals interacting with multiple tags as well as run a longitudinal study to investigate
the robustness of MechanoBeat over time.
We will also explore different oscillator designs to further miniaturize our tags.
Thank you.