hi everyone and thank you for watching
today i'll be presenting privacy mic
utilizing inaudible frequencies for
privacy preserving daily activity
recognition
microphones are perhaps the most
ubiquitous sensor in computing devices
today
with the rise of voice agents computing
devices use these sensors to transform
themselves into listening devices
and understand users through sound for
example these assistants listen to our
verbal requests
and perform daily acoustic activity
recognition tests
like reminding us to brush our teeth or
telling us to wash our hands for at
least 30 seconds
however the ever-present ever listening
microphone presents significant price
concerns to users this conflict leads us
at a crossroads
how do we capture sounds to power these
helpful always-on applications
without capturing intimate sensitive
conversations our devices should hear
activities of daily living like when
washing dishes
but not our sensitive conversations
did you hear that casey got coven are
you kidding i knew going to that party
was a bad idea what was she thinking
i know
unfortunately it's just not that simple
to remove speech frequencies
when we remove speech frequencies from
traditional microphone systems
the baseline accuracy drops from almost
90 percent to around 50 percent
it's because speech frequencies in red
span a large portion of the audible
spectrum in green
from about 300 hertz to 8 kilohertz
fortunately there's a wealth of
information outside of human hearing
by looking at sounds from the sync in
the previous scene we can see that
there's significant information outside
of human hearing
ultrasonic frequencies in purple we
cannot hear at all
to take advantage of these rich
inaudible signals we set out to design a
microphone to take advantage of the safe
and useful ranges but immutably remove
potentially private information
privacy might consist of three major
components a wideband ultrasonic
microphone
in hardware amplification and filter
stage and a low noise
low power high speed analog to digital
converter privacy mic was designed to
maintain the same form factor as a pi
zero
and can be easily deployed as an iot
sensor for further detail
please see the paper
using privacy mic we can capture this
hidden information
to drive activity recognition
applications without the use of any
human audible frequencies at all
so
we can even pick up objects that are
entirely silent to humans like computer
monitors
as the computer monitor turns on watch a
laptop screen showing an fft of the
incoming
ultrasound only data
we formally evaluated privacy mike's
real world performance across three
environments
seen earlier using a 10 round leave one
round out cross validation
and we found competitive results using
our speech filter which only removes
speech frequencies
we found an average whole home accuracy
of 98.9
using our audible filter which uses only
ultrasonic frequencies
we found an average whole home accuracy
of 95.8 percent
overall our real war performance numbers
offer great promise in performing
privacy preserving acoustic sensing
in conclusion our work explored
inaudible sounds as a feature source
for acoustic activity recognition we
designed privacy mic with in hardware
filters to immutably remove
speech or audible frequencies before
digitalization
and we performed a real world evaluation
finding greater than 95
accuracy across all three environments
and filter settings
i wanted to thank you for watching and
please see the paper for complete
details
and additional studies down in this talk