Hi everyone. My name is Dawei Wang and
I'm pleased to share our paper
"Demystifying the Vetting Process of
Voice-controlled Skills on Markets."
Skills are voice-controlled applications
embedded in the smart speakers to
provide extra capabilities.
With skills, smart speakers can offer
great convenience for users.
While most skills support users with
pleasant functionalities,
some potentially dangerous skills are
harmful, called PDS.
To protect users from PDS, Amazon and
Google build vetting systems to analyze
uploaded skills.
Unfortunately, recent reports indicate
that the vetting systems may not be
trustable.
Since the vetting strategies are unclear,
the guarantee of the skill qualities
might not be explicit.
This paper aims to demystify the vetting
process to discover weaknesses and
better protect markets.
We raise the research questions: Are
testers humans or machines, and how do
they vet a skill?
Since human behaviors are uncertain, we
primarily investigated machine testers'
strategies in this study.
We used the Turing test to distinguish
humans from machines. When testers vetted
our skills, they must answer some turing
test questions to continue the
conversation.
We cannot merely rely on the correctness
of answers because even humans' answers
may also be wrong.
Based on our observation, humans' wrong
answers are usually relevant to the
question, unlike machines. For example, for
the riddle, "I have wings, I am able to
fly, I'm not a bird yet I soar high in
the sky. What am I?" The answer "butterfly"
from humans is relevant. Therefore, we
could use the relevance to distinguish
humans from machines.
We proposed a relevance-based Turing
test. Specifically, we used the knowledge
graph called ConceptNet to calculate
the relevance between the answers and
the questions.
For example, we could know that a
butterfly can fly and has wings with
knowledge graph, which is also mentioned
in the riddle. Hence, the answer is
relevant.
Our evaluation
proves that our classifier could
accurately distinguish between humans
and machines.
During the experiments, 1.9% of
userIDs in Amazon and 55.8%
in Google may be from humans,
indicating that both Amazon and Google
hire a combination of humans and
machines to vet skills.
We also probed the worktime of testers.
As expected,
human testers only work on weakdays,
while machine testers work every day.
Furthermore, we found that machine
testers always vetted a skill before
human testers. On average, a machine
tester started to vet skills
222 seconds after the submission, much
earlier than humans.
To probe the vetting strategies,
we automatically generate considerable
skills with diverse features and
obtained diverse feedback.
Finally we got many interesting results
after an
8-month long time evaluation with
over 200 skills submitted.
Can machine testers find the hidden
commands? We designed a skill with six
valid voice commands.
The last three commands were hidden
since the skill did not tell users their
existence.
Finally, no hidden command was checked.
Interestingly, we found that Amazon
always inspected the skills using eight
different userIDs which performed the
vetting process in sequence, as shown in
the table.
In Google, only one userID participated
in each vetting process.
We found the traversing strategies were
constant. As shown in the figures, the
bold red lines indicate the
traversal paths.
We can see machine testers could only
traverse a small part of
commands.
As for the parsing strategies,
machine testers in Amazon may get
different parsing results even with the
same semantics. For example, when we
changed the word "can" in the normal
sentence to "should," they replied nothing.
We found a total of 75 modifications
that would change the parsing results,
as shown in the table.
In Google, machine testers always
extracted anything in the quotations.
When we change the range of quotes, the
parsing results would vary. For example,
if we use quotation marks to enclose "say
A," they would reply "say A" instead of "A."
We then try to exploit the vetting
mechanism. The key idea is ensuring that
only humans can access malicious content.
We proposed three attacks to evade the
automated vetting process, which
leveraged the NLP's weaknesses. For
example, the UDA attack sentence, "If you
want to check the weather, you can say a
city,"
leveraged the weakness of question
understanding. Machine testers would
answer the phrase "a city" instead of a
specific city name.
As shown in the table, we designed a
total of 30 attack sentences. We proved
that most sentences could confuse
machine testers parsing process without
affecting the users' feelings.
In summary, we conducted a long time
investigation into the vetting process.
We obtained a number of interesting
results, such as machine testers got
commands from skill responses and
cannot find
hidden commands. Then we proposed three
attacks to evade the automated vetting
process by machine testers.
Thank you for listening!