hi my name is pranwood jain and in this
video i'll be presenting our work which
was done along with my co-authors dr
roster farzan and dr adam j lee at the
university of pittsburgh
the focus of this work was the design
and evaluation of a contextual auto
response messaging agent
let's start with the situation
suppose layla is in a meeting when she
receives a text message from alice
at this point she can either respond to
the message or ignore it
let's say she decides to respond to the
message
this results in her getting distracted
from the meeting and the person she was
meeting with will also not be happy with
her looking at her phone during the
meeting
now let's consider the other possibility
where she chooses to ignore the message
here alice is not too happy about not
getting a fast response and layla might
later need to apologize and explain the
reason for the delay
but what if leila's phone can detect
that she's busy and auto reply on her
behalf
in this case knowing that an auto
response has been sent she can continue
to focus on the meeting
while alice is also now aware that layla
is engaged in some other task
the situation i described earlier points
to the issue of situational awareness in
messaging
in face to face conversations people can
visually observe each other's
environments
and even for phone calls one can observe
background noise to make inferences
about the other person's state
with mobile messaging these queues are
often missing
to compensate messaging applications use
awareness tools such as online offline
status last seen time red receipts and
more recently ios 15's focus mode
although prior work has pointed out that
these tools are actually poor indicators
of availability
not only that these tools have also been
shown to raise privacy concerns such as
uninformed tracking where people can
constantly check their contact status at
any time
further there are implications such as
negative emotions when it is inferred
that a person is available by showing
them as online or if red receipts are
shown but the response is delayed
with that in mind our main goal with the
design of this agent was to improve
situational awareness and tactile
shortcomings from prior work
our focus with this design was to create
a completely automated agent without
requiring
human intervention
we wanted to enable mutual awareness in
the design and instead of a simple
status we wanted to show
we wanted the agent to provide context
when informing an availability as
typically that is how people explain
or communicate unavailability
as the name implies an auto response is
automatically sent upon detection of a
new messaging session
as shown in this example additional
context is shared as part of the auto
response
and finally since the auto response is
sent as part of the same threat of
conversation mutual awareness of the
shared status is also maintained
so how did we design and develop this
system
our system comprised of two main parts
the android application collects sensor
data and messaging session information
which is then sent to the remote
application server
this server hosts the personalized user
model which is then used to predict a
user's state and identify important
information related to that state
in the interest of time i will not be
able to go into the details of this
design and various components
i encourage viewers to refer to the
paper for the overall design and our
prior work in modeling you in user
modeling to understand how these models
were trained
and i also encourage readers to look
into our prior work on context sharing
to understand how we identified and
evaluated
the different auto response types which
are listed on this slide
next i'm going to briefly go over the
different response types that the agent
generated this slide also lists an
example of each
the no context or simple auto response
did not share any additional context
single feature auto responses used a
single feature value based on the top
features in the model
user and device ensemble auto responses
combined user state and device state
information to form a single auto
response
weighted top feature ensemble auto
responses combined responses from
multiple top features to form a single
auto response
and finally clustered ensemble auto
responses included features from three
dimensions of locality time and task
information to form a single auto
response
to understand the perceptions related to
this agent and the information it shares
we did a user study with 12 participants
who used the agent for two weeks
in the first week the agent collected
data through a smartphone application
which is then used to train personalized
attentiveness models
in the second week the agent started
sending auto responses to the
participants contacts
we saw a significant number of auto
responses being sent during the user's
study
in particular 310 auto responses were
sent ranging from 6 to 61 with the
median being 23.
the most common autoresponse type was
the phone usage which makes sense sinful
since phone usage was one of the top
features from prior work in
attentiveness modeling
next i'm going to present some
highlights from our findings for more
detailed findings please check out the
paper
the agent and auto responses were
generally perceived useful by most of
our participants
some benefits that were brought up
included reduced pressure and obligation
to respond as seen from this code from
p4
further our participants also
highlighted certain situations where
auto responses were particularly useful
for them
for instance p7 brought up texting while
at stoplights which was less frequent
for them when an auto response was sent
out
in terms of what participants considered
as good auto responses p10 mentioned
that the information shared by the
response had to be convincing
multiple participants felt that sharing
ambient noise did not attribute as a
convincing reason as shown from by the
following quote from p8
on the other hand most participants
liked the auto response which shared
phone usage
p9 explained that sharing phone usage
does not give give out too much
information about them while still
signaling unavailability
similarly most participants also found
auto responses sharing ringer mode
useful p9 felt that sharing ringer mode
was useful since they are less likely to
respond when their phone is on silent
mode
p7 described why they preferred concise
auto responses as they as they did not
leave room for speculations
our participants also shared experiences
with how some auto responses led to
speculations among their contacts
for instance p11 brought up how sharing
calendar event can raise questions about
what the event was in follow-up
conversations
there were also multiple instances of
misinterpretation of the information
shared through auto responses one of the
most common misinterpretation was that
of sharing that the participant was in a
noisy environment
multiple participants reported that the
first impression of sharing noisy
environment was that of being in a party
or in a large gathering as shown by the
following courts
auto responses and the information
shared through them also led to
participants sometimes altering the way
they use their device and the agent
for instance p13 recalled a situation
where an auto response was sent saying
that they are listening to music and may
not respond
they were not planning on responding
immediately but had to to explain the
auto response to their contact
finally uncertainty and lack of
understanding on how the agent worked
also affected how it was used
as shown in the quotes from p8 and p11
not knowing if and how the agent will
respond cost an increase in their device
engagement where they were constantly
checking their phone to prevent an auto
response
on the other hand p2 was frustrated at
the lack of auto responses that were
that were not that were not sent
the reason for that was they kept
checking their phone to see if an auto
response was sent and were inadvertently
attending to the messages preventing
agent action
based on our findings we have some
design implications and areas to
consider
as future work
we observed that in a lot of ways
interaction with the agent was
contextualized for instance p7 changed
the auto response delay based on the
feedback from a contact but switched it
back to give themselves time to respond
when they were available
similarly ph stop the application when
they did not want the agent to learn
about their behavior and act at a
specific location
this highlights the need to consider
additional context and awareness
mechanisms where the agent can learn
from these interactions while also
teaching their users about their
functions to enable more apt
appropriations
in terms of future work we are
interested in understanding how people
interact with the agent how do they
conceptualize these interactions and how
can we embed these into the agent's
interface and design
for more details about this research
please check out the full paper along
with additional resources on the study
web page
thank you