Hello, my name is Yeonjoon Lee.
I'm currently an assistant professor at Hanyang University in South Korea.
My talk today will be about "Using sonar
for liveness detection to protect smart speakers against remote attackers".
This work was done in collaboration with Hanyang University,
Indiana University, Chinese Academy of Sciences, and University of Virginia.
Smart speakers such as Amazon Echo and Google Home are popular.
According to voicebot.ai, as of april 2020,
87.7 million adults own a smart speaker.
Also, Amazon stated that there are now more than 100,000 Alexa skills worldwide.
Besides the basic functionalities, smart speakers
also provide critical functionalities. Good examples are
access banking services, send money, open front door.
However, once smart speakers are attacked
such critical functionalities
can cause serious consequences.
Let's see what can go wrong.
As many of you already know smart speakers take voice commands from
the users and perform various activities. The
problem is the fact that smart speakers also respond to voice
commands coming from other speakers. In our homes there are various type of
network connected devices with speakers. However, according to recent
reports, many vulnerabilities are found from such
devices. By exploiting the vulnerabilities in the
network connected devices remote attackers can hack into the
devices and send malicious voice commands.
Furthermore, recent studies show that the voice interfaces of smart speakers can
be cheated with crafted voice commands. To tackle such
problems, we proposed Speaker Sonar a sonar-based
liveness detection system which aims to protect voice interfaces
of smart speakers.
We carefully designed the speaker sounder to meet the following goals:
provide non-intrusive user experience, conduct real-time defense which does not
require additional training time, use hardware
similar to the current smart home devices such as the amazon echo.
The key idea is to do a simple consistency check.
We check whether the direction of the voice command is consistent to the user.
Simply put, we want to make sure that the voice command is indeed coming from the
user. To detect the direction of the user we
build a sonar system. Our system emits an inaudible ultrasonic
sound.
Our system then tracks the user's direction by analyzing the doppler shift
in the reflected signal. It's basically a sonar made with cheap
hardware. But because we made it with cheap
hardware, our approach can only track users
up to 2 meters.
As mentioned, the user can send voice commands.
However, remote attackers can hack into network attached devices
and send voice commands as well.
But now since we have the user's direction
we can compare it with the direction of the voice command and pass
when the direction is consistent with the movement.
When the directions are inconsistent we can simply reject the command.
With our protection in place the attack only works
when the user, the hacked speaker, and the smart speaker is in a straight line
In other words, it means that the attacker needs to know
the location of the hack speaker, the location of the smart speaker,
the location of the user in real time. Without such information
the attack becomes very opportunistic. The system i just showed is built with
inexpensive devices. As you can see from how it looks the
speaker sounder basically imitates popular smart speakers such as the
Amazon Echo. This is the overview of the Speaker
Sonar. Please check our paper for the details.
These are the frequency spectrum examples analyzed in our system.
The blue box shows the spectrum of this transmit signal.
We designed this wide-band inaudible signal to build a Sonar
with cheap devices. We use the wide-band signal because wide-band signals
work better in reverberant environments such as our homes.
Before we actually can get the direction of the user, we process the received
signal through several steps which eventually
ends up with the signal in the red box. We basically removed all the necessary
information and only left the doppler spectrum
to get the user's direction accurately. Please check our paper
for more details. The following is a room that we evaluate our system.
As shown in the table, our system successfully passes
legitimate commands and rejects malicious voice
commands. We also place the hack speaker in different angles from the user and
evaluate our system. As you can see our system successfully
rejects malicious commands that are sent from 20
degrees or more from the user. We further evaluated the
accuracy of common behaviors. As you can see, while larger movements
get well detected, precision decreases when movements are
small from 2 meters or more perpendicular from
the speaker. This is because the doppler shift gets
weaker in such cases. Hope you enjoyed our work. Please check
our paper for more technical details, micro benchmark
evaluation results, interesting online survey results and
discussion. Thank you for your time and feel free to
contact me if you have any questions.