Hi everyone, my name is Yang Gao, currently a postdoc at Northwestern University. I am very
excited to be here to present our recent work "Voice in Ear". It is a spoof-resistant and
passphrase-independent body sound authentication system. In this work, in general, we use body
sound that propagates through the ear canal to verify the user’s identity. we know that earphone
has come a long way, evolving from large and heavy device 100 years ago, then to a small size earbud,
now we are adding more and more sensors. In the past, earphones only play what are manually
transmitted, today, they are able to play smart by sensing the environment and user’s activity
and they have great potential to become indispensable assistants in our daily lives.
Beyond audio streaming, earphones now can do many other things. However, there are still
much information earphone don’t know, for example, they know very little about the user information,
such as user identity. This lack of knowledge is preventing earphones as an indispensable smart
assistant for our daily tasks. I list some existing approaches of mobile and wearable
authentication and their limitations. Some of them are not secure enough, templates can be
stolen and faked, which suffer from spoofing attacks, such as pattern/passcode, fingerprint,
voiceprint. Some of them have relatively low generalizability, which means that the sensor
cost limits its wide deployment to commercial devices, such as vein pattern and faceID.
Now earphones bring us an opportunity to provide a user-friend and secure solution.
There are lots of scenarios that earphone-based authentication that can provide unique benefits.
such as it can unlock the smartphone and app without requiring active operations;
it can also verify the user's identity during confidential communication such as
bank fraud calls. users can do the payment without holding anything in their hands. last but not
least, it can provide user identity verification without removing their masks and gloves,
which greatly enhances hygiene and self-protection in today’s environment. In general, the biggest
advantage of earphone-based authentication solution is convenient and unobtrusive.
Since We know the earphone-based authentication could provide lots of benefits. The next question
is what kind of information is unique and can be used as potential biometrics.
Then let me briefly introduce the body sound transmission and why it is unique. when the user
is speaking, the vocal folds will vibrate and the vibration will propagate through the body,
some portion of energy will reach to the ear via bone conduction and tissue conduction.
we can model the body sound received in the ear canal by the equation,
it contains the original sound emitted by vocal folds and the decay parameter,
which is very complicated and determined by tissue and bone conduction. Therefore, we assume that the
uniqueness of the tissue and bone and also brings uniqueness to the sound transmission process.
Based on the finding, we design our authentication pipeline,
consists of two major part, feature representation part with two-level sound-transmission
descriptors, and authentication modeling. The level I body conduction descriptor is to find
the unique mapping function between the air propagated sound and in-ear body sound by
utilizing a cycleGAN. The second level asymmetry descriptor is to find the unique asymmetry
information of body sound between left ear and right ear by using transfer function estimation.
Afterwards, the features are fed into the authencation models, including a GMM-UBM
based i-vector that is able to handle context-free speech input,
and an ensemble decision tree. Finally,
we use soft voting to fuse the decision from two classifiers to verify the user's identity.
As the experiment, we collect over 100 context-free sentences over 23 participants.
The result indicates, after collecting 75 seconds of speech content during the enrollment phase,
we can achieve less than 4% Equal error rate. Considering we only test 23 participants, we
also want to know the generability of our model, so we plot the EER trend along with the increasing
number of participants, after our model learn enough data from different subjects,
the error rate increase gradually slow at the converging trend, therefore, we believer,
our system have a great potential to be tested on a much larger dataset.
As a conclusion, we proposed an earphone-based dual-level user authentication system named
EarPrint, leveraging the dual microphones to capture and recognize the user in-body
sound conduction features and body asymmetry features.
Compare with existing mobile and wearable authentication
solutions, our work has relative high accuracy and also can be deployed in earphones.
If you are interested in our work, you can check the details in our paper.
Or if you have any questions, feel free to contact us by email.