Unknown: Hi, everyone, my name is Anastasia Ostrowski, and I'm
gonna be talking today about our paper on mixed method long term
robot usage, specifically focusing on older adults' lived
experience of social robots. In HRI, we are very fascinated in
understanding how are people going to engage with these
systems in long term contexts. But how long is long term? In
previous literature, this has been cited as two months, five
weeks. So you can tell there's a variety of different time
periods that this is defined us. So perhaps let's consider
novelty instead. But then when does the novelty effect end?
Previous literature has cited this period lasting two months,
four weeks. And so again, you can tell there's a variety of
answers. We are still very interested in understanding how
these systems can be studied in long term contexts as well. One
of the most common methods that people use are qualitative
methods. And this involves interviewing, diary logs to
capture people's experiences. These experiences and this
qualitative data can also be paired with quantitative data.
And this quantitative data can come from robot usage logs as
well. In the field, there's a lot of interest in further
promoting mixed method approaches that combine both
qualitative and quantitative data such as the way that we've
done it in our work studying a robot in a long term context
where older adults hosted it in their homes. Our process is
embedded in a one year long co design. And we worked with 28
Older adults ages 70 to 94 years of age. The process that I've
shown here is the overall co-design process that you can
learn more about in the Ro-MAN 2021 paper on this project.
However, I'm going to focus on the robot hosting and robot
debrief stages. The robot hosting stage is where older
adults were living with a robot in their home for at least a
month, though several of our participants decided to keep the
robot for longer than one month. And then we have our robot
debrief stage where older adults recapped their experience with
us about a month into the hosting process. The data that
is gathered in this stage is our robot usage logs, and
interviews. Our interviews are our qualitative data, and the
robot usage logs are our quantitative data that makes up
our holistic mixed method approach. So now I'm going to
talk about our quantitative results. Our quantitative
results are divided into three different groups. And these
groups are defined by the length of time that people were hosting
the robot. We have our overall group, which includes all of our
participants with their robot usage logs. We have our under 50
participant group, which is where the participants hosted
the robot for less than 50 days. And then we have our over 50
Group, which is participants who hosted the robot for greater
than 50 days in their homes. When I talk about the results,
I'm going to have the number of actions on the Y axis. And I'm
going to put the weeks on the bottom. We're looking at a two
month period here, as we thought it was very much very
informative to look at the very beginning of the hosting period.
And you're going to see as the legend shows the three different
groups shown on the next graph.
So these are the usage patterns across the two months. And you
can see that it looks like the pattern stabilized into a
consistent usage around maybe weeks two or three. When we did
statistical testing, we found that the only two weeks that
were statistically different from one another for all of the
groups was week one and week two, suggesting that the users
usage pattern had stabilized somewhere in that time period.
We also compare the exponential decay rates to kind of look
deeper into usage in the paper, as well as we look a little
deeper into what interactions are the people actually doing
with the robot. Now I'm going to focus on our qualitative
results. We also have three groups for this particular type
of data. However, they are defined differently than the
quantitative results. We look at the overall characteristics of
the usage pattern to define these groups. We have our
typical usage group, which is characterized by a high initial
exploration and then dropping to a more consistent exploration.
Our low usage group, which is characterized by periods of non
usage, and then our high usage group where they had overall
very high usage and a less significant usage drop overall.
So first, we're going to look at our typical group. And before I
show those graphs, I'm going to walk you through how we'll talk
about them. So our average usage will be on the Y axis. And the
days is on the x axis. And again, we're looking at a two
month period. So this is our first graph is p 11 and p 12.
And our next example is p 29 where you can see the overall
usage trends. Now we're going to go into the qualitative data to
understand what themes were coming up for this group. One of
the main things was frustration, for example, they said, the
dialogue is just a one two kind of thing. That was one thing
that became sort of evident that it would be much more engaging
if you could have a back and forth that had more than just
two elements. So conversation was a key focus here. And this
also played into why the robot wasn't matching their
expectations as much and was tied to memory where users
really wanted the robot to remember what they just said or
a previous conversation. Lastly, they also talked about feelings
of guilt, where the design of the robot made them feel
compelled that they should interact with it, or they felt
guilty that they weren't engaging with it. Next, we're
going to move on to our low usage group. In our low usage
group, here's an example of P 08, P 09, and p 13. We're going
to spend some time going into p 13. a little bit to kind of talk
about these periods of non usage. These were defined by
proactivity and transparency as two themes. They said it went
from sort of cute to creepy, to, I couldn't keep it on for the
whole day, I would take blocks, I didn't like the thing moving
following me in my house. And so they decided to not use the
robot because they couldn't keep it on because of its proactive
nature. For P 13, we also talked to them about using the right
wording. And so what they meant by this was that it was hard at
times for them to figure out what was the correct way for
them to talk to the robot, making them not use the system
as much. For p 09, they also had periods of non usage just to
refresh your memory. This was also anchored in productivity
and transparency. They said Is he recording audio? Is he
recording video who's listening to it? You the researcher told
us you wouldn't be but thanks to Facebook, we don't trust you.
And therefore this aspect of trust really defined their usage
or non usage in this case. Lastly, I'm going to talk about
the high usage. So here's P02, our first example. They use the
robot as a social catalyst, meaning that they like to have
people come over and interact with them and the robot. So they
are really using it as a way to keep their, keep their human
human relationships very strong. They also mentioned this aspect
of feeling of guilt, like p 11 and p 12. They said, I must
confess, I got tired of it toward the end, after I had him
here. I wasn't checking every day, and then I felt guilty. So
in the morning, the first thing I do is say good morning, Jibo.
Our next participant is P 10. And they had the highest usage
out of anyone in the participant group. The other graphs I've
shown you are around 120. This is 450 for the average usage of
this robot, just to put it in perspective.
And so for p 10. They also asked Jibo 1000 questions to really
define their exploration. And they were exploring the robot
and grading it on how it did in the various answers to the
question, and this is a very unique usage pattern as we did
not request any of the older adults to investigate the robot
in this manner. So these are the different usage groups and the
different trends. And what made it really powerful is combining
our quantitative and qualitative data together. But you may be
asking, you know, what have we learned about how to conduct
long term robot studies using mixed method approaches. So our
first guideline that we present in our work is to incorporate
both quantitative and qualitative data to create a
fuller picture of the robot experience as we've done here.
We also really emphasize that you can learn about users'
perspective of the technology before and after living with the
technology long term to further inform what this experience
really means to people. And though we don't talk about it in
this paper, we do have before interview data, as well as this
after data that is discussed here. For our third guideline,
we say that the robot should be easy to set up reliable,
autonomous and tested in the real world before being deployed
into context to help minimize any issues or frustration on the
part of the user. And for guideline four the research team
should be easily reachable. So if there is any frustration, the
participants in the study have a clear direction of how they can
reach out to someone. For guideline five, the researchers
should explain how the system works and what data is being
collected to increase transparency. And for our sixth
guideline that builds on this, the researchers must be mindful
of the ethical dimensions surrounding the work. For
example, the researcher should be mindful to address concerns
around privacy, data collection, transparency of interactions and
other areas. They should also be mindful of people's attachment
to these systems and be careful to design really mindful entry
and exit strategies for the technology being mindful of how
people may create connections to these devices. And I want to
leave you with this last thought about how qualitative and
quantitative usage and non-usage is critical to understand how
humans interact with robots, and how we can better design robots
and the studies around them, so we can really understand these
systems in long term social contexts. With that, I would
like to acknowledge our Co-designers who we couldn't
have done this without them throughout the whole process
we've engaged with them, as well as Daniella DiPaola and Jon
Ferguson for help with acquiring the usage logs and initial
processing of the data. This work is funded by Samsung
Research and in part by the Human Care Robot Technology for
Aging Society program of the South Korean government. And
with that, I would like to say thank you and we look forward to
answering any of your questions that you may have at HRI 2022