Hi everyone I am Jiaxin from Signify Research
today i would like to present our
work the effect of explanation design on
user perception of smart home lighting system
so what would you find as a common problem
is that when the system
become more smart and intelligent when
there involves more automations
users may mistrust misuse and abandon the
system if there is a mismatch between what
how the system behaves and how they think
it should behave
so we do find actually provide an
explanation is a proven effective way to
increase their trust and enhance the system's
transparency
so our focus of the current study is that
we do find there is a lot of studies,
explanation designs in recommender systems
and e-commerce domain
and they focus on choosing among different
alternatives
and what we do in explanation design in the
smartphone context is we focus more on the
action implementation phase which means we
want to explain the current or a specific
decision
and there is only a few work that took a
closer look to end-users
so what we do in our study is in our first
phase we're trying to understand a bit more
of the attitude and expectations of the users
when they receive an explanation
and we
want to understand a bit more about the
factors of the explanation that affects
their satisfaction
in our second phase we use the technology
acceptance model as a base to understand
how receiving explanation can effect
the peceived control, perceived usefulness,
perceived ease of use, and finally their
adoption intention
so we've designed our study in two
different phases
what is our study based on we did design
three different scenarios and three
different types of explanations
the first scenario is about the systems ability
to control your lighting system when you
are not at home
to act as if you are at home
the second scenario is about
to detect your activity and give
you the most suitable lighting
and the third scenario is related to the
system act like your assistant and also
behave the way for a bigger value of you
such as for your well-being or for your mood
from scenario one to scenario three the
required connectivity, complexity, and
the data usage of the system increase
and for the explanation design to all those
scenarios we use the icon, the system
status and explanation as three different
elements and based on the current philips
hue system,
the home page,
to share the notification
and we have three different explanations
designed
the first one is user centered explanation
which means we focus a bit more on on the
hedonic aspect to tell the user the
information
as if there is a personal touch
and the second way is we try to communicate
user more folks on the utilitarian aspect
and we communicate in an effective and
informative way
the third one is the control group we did
not provide any explanation but only the
system status
so in the first study we conducted
semi-structured interviews with ten participants
five of them are internal employees of
Signify and five of them are external Hue
users
and we presented them with all scenarios
and explanations. We used inductive
thematic analysis and we generate more than
five hundred
codes
what we do find is that
without explanation participants are more
likely to assume the system is actually
misbehaving
and the explanation is helpful to
increase their understanding
what is more important is we do find having
control is
of key importance for the user to have
alongside an explanation
and we also find that for users they want
short simple explanations and only appears
at critical times
icons help. and also they want holistic
information so the global explanation as
well as a specific decisions information
so for the second study we did the online
experiment
with more than four hundred fifty
participants
they are all philips hue users we recruit
them through the Hue beta community
and it's a three-by-three between-subject
design with three explanations and three
different scenarios
and for the analysis we use structural
equation modeling for the technology
acceptance model
and we use a two-way between-subject ANOVA
for the explanation time
so this is our procedure for each
participant we actually randomize this
scenario and the explanation so they only
get one scenario with one corresponding
explanation
and we also listed all the constructs that we used
so what we're trying to test is actually if
receiving an explanation would
contributes
to your perceived control and thus your
perceived ease of use and later can
havs an impact in adoption
intention
so we have our hypothesis listed
what we actually find is that,
on the contrary, receiving an explanation
had a negative effect on the perceived
control
and we also find
perceived control mediates and yields a
completing impact
and suggest that although receiving
explanation leads to a decrease in persons
perceived control still plays a positive
effect on the perceived ease of use
so how so would you find two explanation to
to this
so in the discussion we discussed that
there might be one the systems smarter than
i think as users quotes
this means without explanation they
consider a system broken
so
when they find out "the system is actually
smarter!" in the explanation there is a
mismatch between their mental model of how
the system behaves
and updating this kind of old mental model
requires cognitive efforts
a second way to explain this is that maybe
having explanation triggers or
remind the users the notion of control
involve the privacy calculus
procedure and thus the user center realise
ohh you're actually controlling my system
it activating the notion of persistent risk
of the users
and we also find in two of the scenarios
it's statistically significant in that user
preferred system-centered explanation
then the user-centered explanation
and this actually indicates that the users
may see the hedonic aspects of the
explanation as noises or barriers to gain
information. they do not expect a smart home
system to provide a feeling of social
presence
so they like to keep the message in a
precise and informative way
and also the use of the first pronoun
might be inconsistent with with their
expectation to the system
so finally we do find theoretically we
argue that providing an explanation can
have both positive and negative impact we
address the importance and the complexity
of providing control alongside explanation
in the home system and our study also
enriched and verified the hedonic and
practical
communication styles
in explanation design
practically we can
give more recommendations to the future
explanation design especially in related
human AI systems
of course we have our limitations we do
find maybe the hypothetical scenarios
vignette can be not sufficient to capture
the real life decision making process
and we do want to have in the future a
longitude study that was conducted in a
real life environment
so what's the key takeaways
we do find that
first
explanation
has actually a complex effect to the users
in a short term actually may be receiving
in an explanation can create a sense of
lost control
and in the long run it can still might be
beneficial to for them to make decisions
and understand this
and the second it is actually off the
importance to provide users an option to
control
in the home system
and third we argue that in a smart home system
users prefer a communication style that is
effective precise and informative
and here i would like to end my
presentation thank you for your attention