Hi, I am Shuai.
I will present our TOCHI paper Modeling Adaptive Expression of Robot Learning Engagement and
Exploring its Effects on Human Teachers.
Nowadays, robots can assist people in dealing with a lot of tasks.
How to teach robots perform a new task?
Compared with traditional robot programming, Robot Learning from Demonstration, provides
an interactive method for non-expert users to teach a robot by intuitively demonstrating
examples.
A standard RLfD process usually contains two stages: i) a demonstration gathering stage,
where the human instructor shows demonstrations of a task, and ii) a policy deriving stage,
where the robot student learns a policy from the demonstrations.
And finally, the robot will show its learning outcomes.
However, it usually takes a long time for a robot to derive a rudimentary policy, so
it may be hard to show meaningful task outcomes early on.
Besides, robots usually remain static in the demonstration gathering stage.
These two limitations make it difficult for humans to assess a robot’s learning status
in real time.
In comparison, in human–human teaching, human learners usually convey their learning
status by natural engagement, such as gaze, gestures, facial expressions.
So, we try to adopt a similar idea from human learning to robot learning.
We propose a new RLfD pipeline, where in the demonstration gathering stage, the robot shows
its learning engagement to humans.
The engagement expression is controlled by two factors: humans’ ongoing demonstrations
and robots’ current learning status.
Now, let’s take a look at how we design learning engagement
How to express robots’ engagement?
The first is attentional engagement.
We employ attentional signals as a cue to communicate robot engagement, named Gaze Following.
In short, we first determine which part of human movement the robot student should gaze
at.
We call it attention point.
Then, we control the orientation of the robot’s head to direct to the attention point.
Second, we design robot’s behavioral engagement.
Inspired by behavioral synchronization in human-human interaction, we designed Rhythm
Synchrony.
The general idea is to let the robot make some slight bodily motions according to the
rhythm of human actions.
We design three kinds of expressions using robots’ forearms and fists.
And we show that a robot is synchronized with the rhythm of the human demonstration by making
the robot act in the same period as the human actions.
Now let’s make the engagement adaptive to the robot’s actual learning status.
We deploy Q-learning as the robot’s training algorithm and use the entropy as its learning
status indicator.
Please check out our paper to see the details.
Then, we integrate the learning indicator into the robot’s engagement expression and
propose two kinds of learning engagement.
The first one is from zero to one, Z2O Mode.
It reflects the learning status of robots by adjusting the intensity of robots’ actions.
In the early learning stage, when the robot is in poor learning status, its actions would
be slow and powerless.
And with the learning progresses, its actions would become quick and powerful.
The second one is from disorderly to orderly, D2O Mode.
It reflects the learning status of robots by adjusting the degree of chaos of robots’
actions.
In the beginning, the robot’s behavior looks chaotic and unpredictable.
With the gradual improvement of learning, the robot’s behavior becomes regular and
predictable
Now let’s go to the user study.
We explore two research questions.
How will the proposed Learning Engagement affect users’ mental model of robots’
learning progress?
And How will the proposed Learning Engagement affect users’ overall perceptions of the
robot and the teaching process?
For the user study, we developed an online human-robot teaching platform where participants
demonstrated in front of a webcam.
The captured motions drive a skeleton-based avatar to replicate the demonstration in front
of a virtual robot student in a simulated environment.
We select four skills for the human to teach As policy deriving is time-consuming, we did
not train the robots in real-time.
Instead, we first pre-trained the robot policy on the four skills.
Then we sampled five episodes from the entire training curve to represent different learning
stages.
We recorded both first-person view and full-angel view videos for participants’ better observation.
And we developed a website for participation.
Now, let’s look at the study procedure.
We adopted a within-subjects design, and the independent variable is the engagement mode.
Besides the proposed Z2O and D2O modes, we included two baseline modes, Full Mode and
None Mode.
We used Latin Square design to counterbalance the order effects.
Each participant needs to teach four robots.
Each robot will be assigned one skill and one engagement mode.
And each skill requires five rounds of teaching and each round involves three steps
The first step is to watch the skill to teach.
Second, they are required to demonstrate the skill in front of the webcam.
In the third step, they watch an avatar teaching an assigned robot on their behalf in the simulated
environment.
Through these steps, we hope to give participants the impression that they are actual instructors
rather than bystanders.
We recruited 48 participants.
Participants come from different backgrounds and have different prior experiences with
robots.
Now, let’s look at the results.
We answer the first research question.
For the effects on users’ perception of robots’ learning status, we found that in
different teaching rounds, participants’ ratings change significantly in Z2O-Mode and
D2O-Mode.
We then did a correlation analysis and found that users’ perceptions of robots’ learning
status are highly correlated with their perceptions of robots’ engagement.
Our results suggest that users’ observation of the behaviors of robots will affect their
perceptions of robots’ internal status, which is also consistent with the mental model
theory.
For the effects on users’ expectations of robots’ future learning outcomes, we found
significant differences among the four engagement modes.
Note that all the robots actually have similar learning abilities and learning statuses.
But the robots’ different behaviors lead to significantly different user expectations.
We have to be cautious that inappropriate expectations will lead to the disuse or abuse
of robots.
So, designers must carefully design the behavior of the robot or AI system to help people calibrate
their expectations.
Moreover, we measure the accuracy of humans’ mental models, by comparing the robot’s
actual learning progress, with participants’ perceived robot learning progress.
Results show that our proposed two types of learning engagement help participants maintain
a significantly more accurate mental model.
Now let’s explore the second research question.
Here to save time, I will pick some findings to show.
First, participants generally approved of robots’ behaviors in Z2O , D2O , and Full
modes because they could receive meaningful feedback from robot learners.
Qualitative analysis shows that, users tend to apply social heuristics and norms, to judge
the behavior of robots, which is consistent with the theory of, Computers are social actors
Besides, most participants think their demonstration quality is good in Z2O, D2O, and Full Modes.
And it is surprising that they reflect more on their teaching in None Mode, although robots
did not show any feedback.
This result reflects that, users’ perception of their own performance can largely depend
on the feedback given by the AI partner.
Therefore, the feedback of the AI system should be carefully designed according to specific
goals.
For example, if the AI system needs to raise users’ awareness of their own poor performance,
appropriate negative feedback can be leveraged, to prompt users to make timely adjustments.
Besides, properly expressing an AI system’s inability, rather than blindly catering to
users’ expectations, can promote users’ critical thinking.
In terms of user experience, participants prefer to teach students with progressive
feedback rather than those without feedback or with unchanged feedback.
This finding highlights the importance of user experience when designing robot feedback
in robot teaching.
What’s more, we found that the robot’s engagement will affect users' engagement.
Therefore, designers can engage users by adapting the expression of robots.
Besides, participants gave fewer demonstrations in Full mode, so we should also note that
the robot’s engagement communication could seriously affect the interaction workflow.
Based on our main findings, we derive a set of implications for the engagement design
of robots and more broad AI systems.
First, RLfD should be a mutual process instead of a monodrama.
Second, Robot feedback design should avoid introducing misperceptions to humans.
Third, Engagement can serve as a design material, for transparently communicating the internal
status of AI systems.
We also provide several Design recommendations for engagement expressions of AI systems.
For the details, please look at our paper.
Thanks for your attention.