hello my name is Mike Ligthart and I'm a PhD researcher at the Vrije Universiteit
Amsterdam in the Netherlands and I'm working on the Hero project and our aim
is to develop a social robot companion for children with cancer and hero's main
objective is to contribute to stress reduction and we are working on
facilitating supportive relationship between the child and the robot and we
are developing concrete stress reducing interventions. And this talk is about one
of these interventions namely interactive storytelling. Hero has
to operate as autonomously as possible to not unnecessarily increase the
workload of the hospital staff and to achieve this we need a very explicit
behavioral design compared to when taking a Wizard-of-Oz approach and
this is where the interaction design patterns come in. They provide a
structured way to specify interactive robot behaviors in such a way that they
can be validated, shared with the community, and reused and these patterns
together form a pattern library and we are expanding a pattern library with the
design patterns that can be used in the stress reducing intervention during a
co-design session with a Child Life Specialist, someone was responsible for
the psychosocial care of the children, we drafted the concept of interactive
storytelling as a distractive intervention and the more the children are engaged by
and involved in the storytelling experience the more effective the
intervention will likely be and this is why the interaction design patterns we
have developed focus on supporting children's engagement and agency during
the storytelling experience. And agency requires attention, a cognitive component,
and is modulated by enjoyment and affective component and a positive
effect increases the availability of cognitive resources that can be spent
on engagement. It is inevitable that a child disengages and
re-engages throughout the storytelling experience. It is the robots ability to
re-engage the child that is essential for the effectiveness of the
intervention and to allow children to get more
involved with the storytelling activity we intend to increase their agency
following the work of Roth and Koenitz on interactive digital narratives we
breakdown agency, in this context, into three aspects the feeling of autonomy
experiencing the freedom to influence the story and the feeling of competence
feeling competent to influence a story and in this case by communicating with
the robot and affectance the feeling that your influence actually has an
effect or impact on the story. We don't start with nothing we have an
existing pattern library we developed in past work which includes design
patterns for managing a child-robot conversation and this includes a
recognition and repair pipeline that uses automated speech recognition as a
primary input and in this case we use Google dialogflow and it includes a
touch-based repair mechanisms mechanism and today in this video we will add
three new patterns to our library and these patterns focus on supporting
children's engagement and agency during the storytelling experience the first
pattern revolves around making choices about the story not only about
giving children a direct way to influence a story it also
provides explicit problems for reengagement the problem it needs to
solve is to provide meaningful choices while keeping it feasible for the robot
to process autonomously and from a narrative creation perspective
preventing an unsustainable growth of narrative branches for example and to
address these problems we opted for a predefined story with fixed decision
points and hero can keep the initiative by asking
for a decision at each decision point there are three types of decisions
the first is a shallow branch where the child makes a decision that directly
affects the next part of the story but to keep it sustainable all the branches
merge directly after the second is an alternative ending so
at some point in the story the child makes a decision and that has an effect
on the ending they get and finally there are world state variables scattered
throughout the story for example we have a story about a refrigerator and a
certain fruit plays the role of a DJ inside the party in the
fridge and Hero asks what the child's favorite food is and makes that fruit
the DJ in the story. Moving on to the second pattern. It's where Hero
acts out certain scenes from the story it's telling. It uses its full multimodal
capacity for motions sounds and lights to be as expressive as possible and this
supports children's engagement and Hero furthermore invites children to join in
this allows them to be expressive as well which in turn supports their own
agency and the third and final pattern allows children to record on sound
effects for the story so at certain points and during the story Hero
asked for help for creating a sound effect and records that sound effect and
incorporates it into the story and this supports children's agency and the use
of sound effects in general supports children's engagement and this is
especially true if they made them themselves. We set out to validate the
three design patterns with a user study and to not burden the children in
hospital with this first version we asked 27 school children to evaluate the
patterns the study had a within subject design participants were exposed to two
different stories one with all the design patterns which were called the
interactive story and one without which we called the plain story now we
measured engagement and agency for both stories to get insights into the
participants agency we measured the feeling of autonomy competence and
affectance with the player experience of need satisfaction instrument
and because the measures for competence and affections asked directly about the
interaction it didn't make sense to measure them after the plain story right
there was no interaction so only after only for both stories that we measured
the feeling of autonomy the rating of the feeling of competence and affectance
will be compared with an artificial baseline and our paper you can find more
details about that analysis and for them for now we will focus only on the
feeling of autonomy we furthermore measured both the attention and
enjoyment components of engagement to measure enjoyment we asked participants
to what extent they would recommend the experience to their
friends on the five-point scale and to measure attention we created a light
base distractor instrument this instrument is based on the Posner
queuing task and the peripheral detection tasks both are attention
measuring instruments from the cognitive psychology community and we place the
instruments near the robot in such a way that when the participants would look at
a robot the instrument would be in their peripheral view the light would come and
during 3 fixed moments in the story and the idea is that the stronger the
participant focuses the attention on the robot the longer it takes for them
to perceive the light we instructed the participants to push the button whenever
they noticed the light being on and when we also instructed them there
is no need to explicitly keep an eye on the light and there is no consequences
for pressing or not pressing the button other than that the light being turned
off when the button is pressed and the response times here
are indicative of the attention the participants have for hero and if we
look at the results we see that participants significantly enjoyed the
interactive story more than the plain story and demonstrating a combined
effect of the use of the design patterns while looking at the results of the
light based distractor instrument we see that the attention for Hero
significantly dropped after the act in the story when no patterns are
used but when the patterns are used the level of attention are significantly
more stable and significantly higher than when the patterns are not used and
when finally we find that the participants also feel more autonomous
when the patterns are being employed
summarizing the results we have shown with our user study that the design
patterns combined are effective in supporting children's agency and
engagement during storytelling activity in the paper we provide links to our
materials if you are interested in using the design patterns in your own work and
besides the effectiveness of the design pattern we also evaluate the design
pattern efficiency and user satisfaction and you can find the details about this
in our paper where we also share a number of lessons learned and insights
about the child robot relationship the main limitation of the presented work is
that we did not evaluate it with children in hospital luckily we are
currently preparing a study to do just that and but more about that in future
work and as a final take-home message we would like to encourage everyone to
build and share their own pattern libraries because I strongly believe
that with this collaborative effort we can drive the field of autonomous social
robots forward thank you for your attention