- I'm presenting this work on behalf
of my co-authors, Yoo-Sef from Thailand University
and Andreas from the Max Planck Institute for Informatics.
And today we'll talk about SkullConduct
and how we can use audio to authenticate
and identify users of Eyewear computers.
Eyewear computers store a lot of personal
information and data, such as short messages
from friends or pictures from your last
trip with your family.
And making sure that these informations
are not accessed by others is a major challenge
and the main challenge there lies in the
interaction technique, because, for example,
when you use Google Glass, the interaction techniques
are very limited, so you can use speech input,
or you can use gestures or touch input
on the frame of the Glass.
But these methods are not well suited for
authenticating, because while you can't simply
speak your password loud, for example, that
people in the room would notice immediately that.
At the same time, identifying users is important,
for example, to personalize the applications
on a device such as Google Glass.
For example, the surgeon wants to do surgery.
He wants to make sure that every application is
aligned in a way that he's used to
and not the way his colleagues have aligned
the applications, because they share the device.
And that is a huge challenge we want to address
with our project.
How do we do that?
We use audio and some sort of finger print and
similar to the work of Daniel, we want to use it
implicitly, and how do we do that?
We use integrated bone conduction speaker,
that's basically a speaker that use not
the air to transmit the audio, but the bone
or the skull of the user, and integrated microphone.
And what we are doing, we are playing back an audio file,
and at the same time, when if
the audio travels through the head of the user,
we record it in the front with a microphone
and as you can see the signal changed,
so we call the characteristic frequency response.
And that's because the skull of the user
modulates the signal and since every
head or every skull is different,
with regards to the size or the composition of the
different parts of the skull, such as
the bone or the skin, this response is unique
for a specific person.
And in order to explore this idea, we
developed a system that consists of Google Glass and
an application.
The application consists of four different parts.
A playback part that basically plays back an audio file
and a recording part that records the same file
immediately again and then
feature extraction we used for that male frequency
capture a coefficient that mainly known from
speech recognition or speaker recognition
and that's a classification step,
so a step that identifies users based on the features
we used a nearest neighbor approach.
We conducted a data recording session.
In total we invited 10 participants and
we recorded 10 recordings from each participant
and we did that in two sets of five recordings
and in between the users took off the glasses
and took them on again, so that we
dealed with the different placement of the Glass.
When they put on the Glass again,
they're probably slightly different place
so that we can exclude that as source of the variance.
In total, each recording was 23 seconds long,
and we used the white noise signal,
because it contains all the frequencies evenly.
And we did the study in a very controlled setting,
so we used a quiet room, nobody else except
the participant was in the room,
there was no technology except the Glass in the room,
no laptop, and so on, so that we just have
a controlled setting with the noise generated by
the Glass and its recording.
Okay, looking at the results.
That's a Power Spectral Density diagram
that plots basically the energy that's in the signal
for each frequency that
two recordings from two different users.
The red and the blue is a recording
before and after taking on and off the Glass.
And as you can see, slightly, the signal changed
when taking on and off the Glass,
but the main structure's roughly the same.
But, the difference between two users
is way bigger, and that's what we basically used
to identify and authenticate users and
we did that using a 10-fold cross-validation.
We removed one participant that served as
an attacker in this case, and we removed one set
of another participant we want to identify
or authenticate.
And as a training set, we put the other set
of the recording from the particular user
we want to identify, and the training set,
in addition to the eight other participants.
And our results show that we can identify users
with 97% accuracy,
and for the authentication we calculate
an equal error rate, the rate was the
true acceptance rate and the fourth rejection rate
was equal of 6.9%.
Furthermore, we looked into how long
such a signal should be in order to serve us.
A biometric and we found out that
the signal that's one second long
is long enough to do
the authentication, identification process.
Signal that are not that long will
result in a lower equal error rate.
Okay,
we did the study in very controlled settings,
so there was no background noise,
which definitely would influence our results.
The next step is conduct a study in a crowd like
you can see in the picture.
Does our approach work in the same environment,
or does it just work in the lab setting?
That's definitely something we need to do.
We need also to improve the algorithms we use
off the shelf and wants to see algorithms
that are not directly built for our approach,
but for something similar and we just adopted it.
Furthermore, the question is whether it works for
broader audience, so,
does our approach work with hundreds of thousands
of users, or is the uniqueness of the scan
not good enough for that.
And last but not least, the audio file
we used in the study, a white noise signal
of 23 seconds, which is not that pleasant to hear to.
We think about, especially as we found out
that one second is enough if we can use
something like the feedback you get when you
interact with Google Glass, for example,
the plongs and the sounds you see
when you send a message, is that enough
in order to authenticate or identify users.
That's basically it.
As a take home, our research, SkullConduct
is a biometric system that uses bone conduction
of sound to identify and authenticate users
on eyewear computers, so thank you very much
for listening and I'm happy to take your questions.
(audience applauding)
Questions.
- [Audience Member 1] Hi, Florence Shop from Cunningnum.
Cool work. - Thank you.
- [Audience Member 1] One question I have is
does the placement of the microphone play a role?
Google Glass is obviously not built
for this kind of application, but it still
works really well, so I'm wondering
if you place the microphone closer to the speaker,
does it make a difference?
- I think it, our study,
doesn't make a big difference because it's
not a noisy room, it was completely quiet,
but if you go into like a crowded setting
it definitely can help to increase accuracy
when the microphone points directly at
the audio source, basically.
- [Audience Member 2] Hi, Lania Noo-Chee of Five Networks.
Two questions, one is have you thought of
playing around with the different frequency
of sound, one that wouldn't be
even audible to the human ear, so it
wouldn't be as unpleasant, and then two,
similar to his question with the microphone,
are you concerned with
someone else being able to record
what the sound ultimately is after
it is conducted through the skull?
- Mhmm, so first question yes,
we've thought about it but we didn't do it,
unfortunately, but it's definitely something we
want to look into in the future, so,
yeah, one approach would be a sound that
you hear anyway, the other one would be
to use sounds you can't hear, so it's
not that unpleasant compared to white noise,
and your second question, I actually didn't
thought about that.
But I think we need to figure out if that works,
if you can simply record it from distance.
The bone conduction speaker is built in a way that
the sound should be private in terms of
that others can't hear it.
Obviously you can hear it, otherwise
it wouldn't work, but it's not that loud,
so you can't basically record something like that
from distance of a couple of meters.
You have to be close to do it, but, yeah,
we didn't try that out, but we definitely will.
- [Audience Member 2] Thank you.
- Thank you.
- [Audience Member 3] Hi, very interesting stuff.
Steward Chechter, Microsoft Research.
Do you have any sense for how much
you're detecting bone conduction versus
things that may be more variable over time,
like the other materials that are
in the skull, if someone's congested,
they're gonna have more material back there.
If someone's wearing an earring,
if someone's wearing hair gel that might be between.
Are you collecting data from different sessions
where people might have been dressed differently
and had different hydration levels and so on.
- Definitely, again, something we should look into.
We didn't do that, we did the session at the same day
and so both recording sets.
My feeling is that
the bone conduction speaker is very closely
to the head and I think the influence of
as long as you have not too long hair
should not be that big,
but yeah, again, I have no data to
support my statement.
- [Audience Member 4] Hi, thanks for the presentation,
my name is Vikram from Open in the City UK.
My question is why bone conduction
for sound for authentication, I mean,
is it better than the tactile input,
the normal tactile input or just making patterns?
Like over the skin or something?
- It's an implicit way, so the scenarios
in which you can't interact with your device
and it's a way that
prevents others from
knowing your password, because
you just stand there and do nothing and
it's very hard to even know that
you could get authenticated.
- [Voiceover] Okay, great job.
Thanks, appreciate it. (mumbling)