Hi, my name is Rachel Phinnemore and I'm here to talk
to you about some research I did while I was a
master's student at the University of Toronto on voice assistants.
Have you ever found a voice assistant to be creepy?
In this work, we investigate the factors
that make voice assistants be perceived as
creepy through a three phase process, including
construct, definition and item development.
Secondly, scale development and finally scale evaluation.
We produced a seven item PCAS scale which enables
developers and designers to measure the perceived creepiness of
voice assistants and thus ensure that newly designed voice
assistants do not introduce these perceptions.
You.
So to start off, have you ever interacted with
a voice assistant and felt one of these things?
That it responds unexpectedly?
Perhaps it makes you feel like you're being
recorded or gives you an odd response?
Well, in case you've not experienced any of these
things, let's take a look at a recent Amazon
Alexa commercial that shows how voice assistants could potentially
be perceived as creepy in the future.
Hey, babe, check this out.
Alexa, it's game day.
Streaming football on prime video, closing blinds.
Chilling Rose Rose.
Well, it's an afternoon game.
It's like she can read your mind.
Your mind.
I love that we get to sleep in
Ordering fresh mint mouthwash, extra strength.
I was thinking I should get a
spray tan because that's on Wednesday.
Activating blender
Funerals on Monday.
What about the gold, Papa?
Can't you see that treasure? All along?
It was here.
Love the eye patch.
When is the show open?
March 8.
Setting reminder to fake your own death on March 8.
What the when you have to do
those love scenes with hot guys.
Is that fun or is that like the worst?
The worst.
This bread is delicious. You make it. Yes.
Guys from my gammy's recipe. Announcement.
Gammy is short for she bought it at Whole Foods.
Announcement.
Colin left the oysters in the car for 5 hours.
It's probably better that it can't read your mind.
Bad idea.
What's the game?
So, because voice assistants can be perceived
as creepy, how can we ensure that
future voice assistants we design are not?
Before we dive into this question, it's important
to define what we mean by creepiness.
As defined by Langer and Konig, creepiness
is a potentially negative and uncomfortable emotional
response paired with perceptions of ambiguity towards
a person, technology, or situation.
Further, it's also important to define voice
assistant as defined by Ermolino and Tiberius.
A voice assistant is an artificial intelligence powered
computer system that aims to imitate human intelligence
while engaging in realistic conversations with users.
However, we haven't been the first
to focus on research and creepiness.
In 2016, there was a paper on the nature of creepiness,
which produced a survey on what makes a person creepy.
In 2018, there was the Creepiness of
Situation Scale, which identified the role of
emotional creepiness and creepy ambiguity.
And in 2021, there was the Perceived
Creepiness of Technology Scale, which identified the
role of implied malice, undesirability and unpredictability.
So while there have been these prior
works on creepiness, voice assistants have unique
differences from GUIs and other technologies, including
their always on nature and anthropomorphic effects.
So we ask, how can you
measure creepiness in voice assistance?
To do so, we use a three phase
process from Boateng and all in 2018.
We start with the construct definition
and item development, where we reviewed 167 papers, extracted
381 quotes, and three researchers ran an open coding
process to produce these 51 items.
We then conducted four expert interviews who gave
feedback on the 51 items as well as
for additional items, which led to 88 items.
We then took these 88 items and ran exploratory
factor analysis, where 198 participants read a creepy scenario
and then rated the scenario using these 88 items.
We then conducted Varimax rotation and removed the low
loading items to produce the seven item scale.
We ran three subsequent surveys in the scale evaluation
to ensure the validity of the scale, and further
details of this can be seen in the paper.
This process led us to produce the validated
seven item PCAS scale, where the items correspond
to the five factors of Control, Intention, Privacy,
Behavior and Value, with items annotated with the
asterisks being novel creepy factors.
So now that we've outlined why the PCAS scale was
important, how it was developed, and how it was developed,
let's discuss how the PCAS Scale can be used.
The PCAs Scale is designed for users
who have experience with voice assistants, such
as issuing a voice assistant command.
Users will view or use a prototype of a voice
assistant and then complete the seven scale items using a
one of disagree to seven of strongly agree Likert scale.
The responses will be summed with scores
ranging from seven to 49, where higher
scores indicate greater levels of creepiness.
In addition to the scale itself, we produce
a set of design guidelines that correspond to
the scale items under the guidelines of intention.
For example, we provide suggestions such as
offering transparency about device functionality to prevent
false mental models from being created, as
well as using a multilayer interface approach
to overcommunicate the functionality with explanations.
In summary, the contributions we make are a
survey tool that measures the perceived creepiness of
voice assistants, the identification of four novel factors
that influence the perception of creepiness in voice
assistance, particularly control, privacy, behavior and value and
finally, design guidelines to improve the design of
future voice assistants.
Thank you very much and hope you have
a chance to check out the paper.