Hi everyone, my name is Alyssa.
I'm a 3rd year PhD student at UC San Diego.
Today I'll be talking about JESSIE, a robotic system we developed to enable novice programmers
to program social robots be expressing high-level specifications.
15-20% of the world's population has a disability, and this number is only predicted to increase
as the world's population grows older.
However, the number and availability of full-time caregivers to support this population is limited.
Robots are a promising means to fill these care gaps, particularly social robots deployed
in a person's home.
For instance, Mabu is used for medicine management, and Paro has been shown to reduce stress among
people with dementia and their caregivers.
It is critical that these robot interventions are personalized to the individual in order
to maintain engagement and adherence to home treatment plans over long periods of time.
Healthcare providers, patients, and informal caregivers need the ability to easily program
and personalize robots to suit the individual's needs and goals.
However, programming robot behavior is hard.
Even with advances in end-user programming to support novice programmers, these approaches
are typically low-level and procedural.
They typically require the programmer to specify each individual action such as what to say
and how to move, and must be extensively tested or else they run the risk of having unintended
consequences on potentially vulnerable populations.
Thus, we developed JESSIE, which we apply to our own research on mild cognitive impairment (MCI)
and dementia, to help novice programmers quickly and easily program social robots to
exhibit complex behaviors.
We couple existing control synthesis methods with an accessible high-level specification
interface to enable users to specify and synthesize personalized activities, reactions, and behavioral
constraints.
Thus, users can focus on their overarching goals rather than specific implementation
details or individual robot actions.
Here's an overview of how it works.
First, programmers can specify high-level behavior such as constraints and reactions
using our tangible language.
Notice that each card represents a different behavior and programmers can lay them down
in whatever order they like.
Orange cards represent conditional statements.
Then, JESSIE automatically generates and executes a robot controller using LTLstack, an open-source
Linear Temporal Logic (or LTL) synthesis and execution tool.
Finally, the robot executes the desired behavior and reacts to users in real time.
KURI: Hi there! It's great to see you.
Let me know when you are ready to start.
Great!
Let's get started.
Let's start the number game.
Let's get through 15 numbers on this list.
Here we go!
4.
5.
On that last list, you got 86% correct.
Great job, I knew you could do it!
Thank you for spending time with me today.
I will see you next time.
We evaluated JESSIE with six neuropsychologists, its envisioned end-users who will be personalizing
robot treatment regimens.
They had no prior experience programming robots.
We taught them how to use the system, then they built an interactive session for someone
with MCI.
After creating the treatment session, they watched the robot execute the treatment they
designed.
We then interviewed each neuropsychologist for qualitative feedback on the design process
and execution.
We then employed a grounded theory approach to find main themes from the interviews.
Overall, we found that by improving the accessibility of control synthesis, our system enabled clinicians
to successfully program at least one interactive session for a person with MCI.
They also reported high usability and learnability of the system, and described using it as easy,
simple, and straightforward.
One main theme that participants identified was increased ability to personalize the cognitive
treatments.
Personalized sessions are critical for people with MCI because their needs and goals can
change as their condition progresses.
Participants stated that feedback style can significantly impact the user's
recollection of different cognitive strategies and how they apply them to their lives.
Thus, they suggested personalizing feedback type and frequency.
Participants also requested control over the robot's communication modalities to personalize
based on the user's sensory abilities and personal preferences.
For instance, the robot may provide more visual and verbal cues for older participants.
They also suggested that the robot sense different behaviors about the user to infer their state,
such as sedentary time, social activity, and mood.
Finally, it is critical for the robot to adapt to the user, especially as their preferences,
cognitive abilities, and moods change over time.
One participant suggested giving encouragement depending on the user's performance with the
activities.
This will encourage them to stay engaged and maintain consistent interaction with the robot.
Additionally, MCI can be progressive, so people's needs, goals, and abilities can change over
time.
Thus, participants indicated that it would be beneficial to vary the status of the robot.
For instance, depending on the stage of the MCI, clinicians may have different goals for
the robot, such as monitoring in the early stages and providing education or intervention
delivery during later stages.
Participants also noted that the robot may take on a more passive approach while the
person with MCI is more independent, such as by observing their behavior to establish
a baseline.
As the MCI progresses however, the robot may need to take on a more active approach and
initiate interaction with the user.
Additionally, many participants work with people with MCI across both clinical intervention
and research contexts.
The goals and roles of the robot may change significantly in each of these contexts, so
clinicians wanted a way to easily create and switch between modes of the robot.
Finally, collaboration is an important aspect of cognitive training, as clinicians and the
people with MCI work closely to identify goals in training, and how to address them.
Participants wanted ways work together with their patients to support their goals, such
as by setting reminders and planning activities that align with their needs and interests.
They also suggested the ability for clinicians to set high-level goals and let the user select
specific activities to practice those skills.
Additionally, the clinician will not always be with the person with MCI when they interact
with the robot, so they suggested implementing ways to enable the user to work directly with
the robot to develop and assess their goals.
Finally, people with MCI may be working with multiple healthcare providers, which our participants
were mindful of.
They suggested allowing for multiple providers to collaborate and program the robot for a
single person.
In our discussions, participants raised some crucial questions for personalized, longitudinal
HRI that need to be explored further moving forward.
Participants mentioned how the person with MCI may see the robot as a companion as they
complete cognitive training.
This raises the question of what the robot's role should be in the relationship between
the clinician and people with MCI, such as companion, point of connection, or personal
assistant.
Our participants also identified multiple levels of timing they wished to control such
as scheduling trial-by-trial feedback, feedback after numerous sessions, and setting the duration
of different activities.
Additionally, our study illustrated that robotic systems need to support multi-party programming
so that multiple stakeholders with different goals and backgrounds can program the robot
at various points throughout its deployment, including neuropsychologists, the person with
MCI, family members, and other clinicians.
Programmers also need to be cognizant of the user's cultural background and how they impact
their preferences, such as communication style and modality.
This can significantly impact the user's adherence to treatment plans.
Finally, many ethical considerations arose in our discussions, as people with MCI are
a vulnerable population.
For instance, participants imagined that the robot could monitor the person in their homes
and produce compliance reports.
While this has clinical benefits, it also raises privacy concerns particularly for people
with more advanced MCI or lower technological literacy.
JESSIE also has a number of contributions for the broader HRI community, beyond just
neurorehabilitation.
First, as JESSIE is an artifact to support reproducibility in HRI and robotics research,
we have made the software and materials open-source so that other researchers can use JESSIE by
simply modifying our existing behaviors or creating entirely new ones for many different
applications.
Thus, JESSIE is easily extensible to other robotic systems and applications beyond MCI.
To demonstrate this, we replicated and executed programs that clinicians produced on a TurtleBot
and provide instructions on how to do this in our open-source materials.
Additionally, It is the first end-to-end system to make dynamic control synthesis accessible
through a tangible interface.
This enables anybody interested in programming robots to create rich personalized behaviors
and reactions without extensive training or testing.
In this way, JESSIE is able to support the whole ecosystem of stakeholders, particularly
those without prior programming experience.
Robots have shown great potential to support people with MCI, and JESSIE will extend the
scalability, accessibility, and personalization of social robots.
The system is open-source, available on GitHub for anybody to try and extend to their own
platforms and contexts.
Thank you for your time.