Hey there my name is AndrĂ© Pereira and I'm presenting our work on the effects of
different interaction contexts when evaluating gaze models in HRI.
this is joint work with Catharine Oertel, Leonor Fermoselle, Joe Mendelson and Joakim Gustafson
if social interaction partners are mutually aware that they are
attending to a common target that phenomenon is generally described as
joint attention which is the focus of our work this usually involves several
phases mutual gaze is used to establish attention then referential gaze is
used to draw attention to an object of interest the other party then typically
responds and establishes joint attention
and finally mutual gaze is established again to ensure a shared experience
in HRI joint attention has shown that
it can make cooperation with robots more efficient for example a robot in the
factory assembly line should ideally perceive collaborators attention and
also provide cues that demonstrate its own attention and what it is going to do next
social eye-gaze behavior generation is
also important for creating the illusion of human-like appearance and behavior
which is the focus of our work we previously introduced a responsive joint
attention system that perceives multimodal information from users and
can communicate joint attention via controlling a robot's gaze an initial
evaluation of our system in a spatial reasoning task with adults showed it to
improve users perception of the robot social presence to investigate the
repeatability of our prior findings we conducted two further studies that
employ the same gaze system with the same robot and the same task but in two
further contexts we wanted to find out if results from a situated first-person
perspective generalize to observers watching videos from a third party
perspective and also if we can generalize our initial findings from
adults in the lab to children interacting with the robot in a school
environment to study our gaze model we created a task where a robot can
collaborate with participants
Robot: You are doing great
the participants' goal in the task is to visualize a 3d cube structure and reconstruct it in 2d
Robot: Are you really sure?
Participant: Can you repeat the question?
Robot: okay, I would probably remove this yellow piece
a semi-autonomous wizard interface is used to control the robots high-level behaviors
TTS echoing participant: tell me the limitation again
Robot: in this second round we must use at least three squares in line but not more
participants have to complete three rounds with increasing difficulty
Robot: this arrangement does not really look correct
so how does our gaze system work it is composed of a two layer design a
proactive gaze layer that initiates joint attention and a responsive gaze
layer that responds to attention cues it can simulate joint attention in any
scenario where a robot interacts with the user in a certain environment while
collaborating in a task each layer contains different modules and starting
in the proactive layer a decision-making module controlled by a human wizard
selects when the robot should perform a dialogue act when dialogue acts are
targeted at a user the robot attempts to establish eye contact and when dialogue
acts are targeted at task objects the robot attempts to initiate joint
attention towards those objects the idle gaze module controls the robots gaze in
moments of inactivity idle gaze shifts occur when the speech content associated
with a dialogue act finishes or the timer for any previous gaze shift ends idle
gaze can target users objects in the task or looking at a random point
in the environment which makes it seem that robot is distracted or averting
its gaze the responsive gaze layer uses real-time multimodal perception to
minimize the robots idle time and increase the usage of more meaningful
responsive joint attention behavior our gaze tracking module enables the robot
to look at approximate positions of the tasks that the users are looking at and
it can also look back at users if they shift their gaze towards the robot this
is essential to create a feeling of responsive joint attention as when
trying to establish joint attention users tend to start and finish with
mutual gaze object tracking detects when a user moves places or remove a puzzle
piece from the board here the robot often assumes that the piece is the
current focus of attention and can of course gaze at that location
our last responsive module is speech activity when speech activity is
detected and the robot is not engaged in a higher priority gaze command the robot
attempts to establish mutual gaze with the user to simulate listening behavior
for extended details on how the gaze system works please consult the original IROS
paper and our github project page we evaluate our joint attention model in
two different conditions the control condition represents a typical HRI
method of most robots and only uses the proactive gaze layer with
decision-making and idle gaze we carefully designed the idle module to
exhibit believable gaze behavior with the goal of making a fair comparison
with the full condition the full condition is as the name says and uses
the full model that we just described with both proactive and responsive layers
our hypothesis is that adding this extra responsive layer will result in the robot
being perceived as more socially present by looking at our objective data and
matching the gaze patterns of the users and the robot we look into measures of
gaze synchrony and we also look into the subjective measure of social presence
social presence is defined as the sense of being together with another we are
using the network minds measure of social presence that is divided into six
dimensions with two distinct directions the bi-directional nature of this
measure makes it a sound choice for evaluating the effects of each type of
joint attention individually initiating joint attention and responding to other's
joint attention in this paper we look into the perception of the robot
direction as we manipulated the presence of responsive joint attention behaviors
the first replication study we performed had the intention of testing whether
third party observers could perceive the full condition as more socially present
we created a crowdsourcing task where in one video the robot was running the full
condition and in the other the control condition the same actor was responding
similarly to the robot in both videos and both the actor and the robot
displayed the exact same verbal behavior crowd workers viewed each video
separately with a length of approximately one minute and 50 seconds
and could rewatch the videos as many times as necessary to reply to a set of
forced choice questions this study suggested that external observers do
perceive social presence differences between our conditions as they consider
the robot in the full condition as more aware of its interaction partner more
capable of understanding messages and more emotionally involved with the user in
the task now we will compare our two real-time studies where participants
interacted with a robot in real time in the original study with adults we
observed positive significant differences in message understanding and
in the total social presence measure we expected that when replicating our study
with children they would be more engaged with the task and it would result in
even clearer findings in support of our hypothesis although children rate the
robot significantly higher in our social presence items the study revealed no
significant differences in the social presence measures between conditions
despite not reporting differences between our gaze conditions children
when compared to adults found the robot to be more helpful would be willing to
cooperate more with the robot in the future and like and trust the robot to a
greater extent in both conditions either because the task or the robot was more
compelling for children or because of their tendency to please the
experimenters most of the children scores approach the upper bound of our
Likert scales this might have made it more difficult to find differences
between our conditions regarding behavior synchrony we can observe a
significant difference when we compare the amount of joint attention in both
groups overall children exhibited a greater amount of joint attention these
differences can be explained by the fact that children looked much more at the
board which gave the robot more opportunities for establishing joint
attention and children looked more at the board because they exhibited more
difficulties this is confirmed by our direct question that asks participants
how difficult they found the task and by several of our objective measures that
point to younger participants having more difficulties in completing the task
to conclude the results from the two replication studies reported in this
paper suggest that our main hypothesis was supported with external observers
but not with children our findings suggest that the HRI
community should take into consideration the selection of an appropriate task and
interaction context that provides significant opportunities to observe the
robots behaviors while we initially thought that our task would be more
engaging for children and thus more appropriate to evaluate our gaze model
it appears that external observers and participants that established increased
levels of mutual gaze with the robot were more successful at detecting
improvements in our manipulated behavior children had greater difficulty with the
task which reduced their attention to the robot and in turn might have reduced
their ability to perceive the robots responsive joint attention behaviors in
addition to providing further evidence that supports our initial finding with
adults the results from the observer study afforded us with the confidence to
perform additional real-time studies although external observer studies can
often be more limiting due to participants not being situated in the
interaction they can have the advantage of being designed in a way that users
are already visualizing the target robot with the target behaviors which can
provide very accurate preliminary evaluations and this concludes my
presentation and thanks for your attention