Hello, I'm Gordon Briggs, and I'll be presenting collaborative work
done between the US Naval Research Laboratory
and the Human Robot Interaction Laboratory at Tufts University.
Our talk is entitled A Novel Architectural Method
for Producing Dynamic Gaze Behavior in Human Robot Interactions.
Gaze is a powerful cue.
It can signal who or what an agent is paying attention to. An agent's.
gaze behavior can also signal lack of attention or increase mental workload.
Because we constantly interpret gaze, both
the presence and absence of a dynamic gaze behavior, it's important for robotic
agents to send the right signals with their gaze.
To illustrate this,
let's take a look at an interactive object learning scenario here.
A Fetch robot is using
the DIARC Cognitive Robotic Architecture's one-shot learning
capabilities to learn the names of novel objects in its environment.
The red object is a "glorp"
the blue object is a "blicket."
"Okay,"
point to the "glorp."
While the robot has demonstrated that it has correctly learned the objects,
you probably weren't entirely sure of this
until the robot pointed to the right object at the end of the interaction.
Despite the robot giving affirmative
verbal feedback, the lack of gaze shifts during the dialogue interaction may lead
to uncertainty about what the robot is actually learning.
Because the importance of gaze
in facilitating social interaction, including interactive learning,
has long been recognized, a variety of prior computational
approaches to generating gaze behaviors in HRI contexts have been developed.
In a survey written by Admoni
and Scassellati, three categories of prior approaches were identified,
including data driven, heuristic, and biologically inspired approaches.
Data driven and heuristic approaches are
trained or tailored, respectively, for specific tasks and robotic platforms.
In contrast, biologically inspired approaches use models of human perception
or cognition as a basis for generating gaze behaviors.
While these model driven approaches tend
to be more generalizable, they still suffer from some shortcomings.
Some model driven gaze behavior approaches
are based on either fine grained models of visual attention or broad models
of human cognition that don't necessarily have dedicated attention mechanisms.
Others have explicit attention mechanisms and have been demonstrated in robotic
domains but do not have natural language communication capabilities.
Here, we propose a novel model driven approach to generate gaze behavior
in human robot interactions, including human robot dialogue interactions.
To do this, we developed an integration
between two pre existing architectures, the DIARC Cognitive Robotic Architecture.
Primarily developed by the Tufts University HRI Lab
and the ARCADIA modeling framework developed at NRL,
the DIARC architecture provides a flexible basis to interface
with and control a variety of robotic platforms, as well as other capabilities
such as natural language communication and 3D processing phase faculties.
Arcadia provides a cognitively inspired framework that has attention as a central
architectural mechanism, which can then be used to guide a robot's gaze direction.
The aim of the DIARC/ARCADIA integration is to provide a task and platform agnostic
means to drive robot gaze behavior, factoring in both bottom-up perceptual
and top-down cognitive considerations, including natural language dialogue.
As ARCADIA, unlike DIARC, is relatively unknown in the HRI community,
we wanted to take a moment to discuss it further.
Arcadia operates in discrete cognitive cycles.
On each cycle, a series of components produce informational elements based
on input from sensors or the previous cycle's informational buffer.
Additionally, Arcadia uses an attentional strategy to select a single informational
element to be the focus of attention, which is broadcast to all the components.
Components may or may not engage in information processing differently
depending on the focus of attention, allowing Arcadia to model both attention
dependent and attention independent perceptual and cognitive processes.
For instance, Arcadia has been used to build fine grained models of a variety
of human psychological phenomena, including perceptual tasks such as
multiple object tracking and tasks that begin to bridge both perception
and cognition, such as number perception and causal judgment.
To integrate DIARCA and Arcadia,
we wrapped an instance of Arcadia in a DIARC component.
DIARC could then supply information
directly to Arcadia's informational buffer.
While DIARC is not overall
a synchronized architecture, each direct component does have its own update cycle.
Thus, the Arcadia cycle could be linked
with the update cycle of the direct component.
On each cycle, the direct component could retrieve Arcadia selected focus
of attention and, depending on what sort of informational element this was,
DIARC could shift the robot's gaze direction to orient itself
accordingly.
We developed a proof of concept integration demo in which 3D object
information from DIARC's vision system was made available to Arcadia.
We implemented a simple attentional
strategy, which inhibited objects in the center of the robots field of view.
An uninhibited 3D object would be selected instead.
Here, we present a video showing the ARCADIA/DIARC integration in action.
In this demo, multiple objects are in front of a PR two robot.
DIARC recognizes these and forwards 3D object information to Arcadia.
As mentioned previously, Arcadia's attentional strategy
in this demo is configured to perform a simple inhibition of return behavior,
deprioritizing objects that have already been fixated on.
An active shifting of robot gaze between
objects emerges from the simple attentional strategy.
Next, we sought to demonstrate linguistic cuing behavior.
To do this, we injected additional
information from DIARC's dialogue system into Arcadia's informational buffer.
Next, Arcadia components could then
process semantic content from the latest utterances received by the dialogue system
and use this in a new, more complex, intentional strategy to bias attention
toward objects with semantic similarity to recent descriptions.
Here, we present a video showing how this
new Arcadia direct model can improve the interactive object learning
interaction shown at the beginning of the talk.
The natural language dialogue, 3D visual
processing, and object learning functionality in DIARC is identical
to the interaction presented at the beginning of the talk.
"Do you see the red object?"
However, as you can see,
the addition of the Arcadia integration component and use of Arcadia focused
attention to guide the robot's gaze behavior provides additional non
linguistic feedback to the instructor or any observer.
"The red object is a Glorp."
"Okay."
"The blue object is a blicket"
"Point to the Glorp."
You can ask yourself whether you feel more confident that this version of the robot
has correctly learned the object compared with the one at the beginning of the talk.
Having demonstrated the integration in action, what are our next steps?
First, we've already begun to use our integrated system as a platform
for examining the utility of gaze in interactive learning.
Some recent work by other groups for some presented last year at HRI,
have examined how gaze behavior affects judgments of the robot's attention
to learning and confidence. Both to evaluate our integration
and to develop a more fine grained measure of utility of gaze in learning scenarios.
We wanted to ask people the question we just alluded to before:
"Do you believe that the robot has correctly learned the objects?"
To this end, we conducted a pilot study which we described in our paper.
We also wanted to use our integrated architectures to develop more complex gaze
behaviors in a variety of hire scenarios, for instance, developing more natural gaze
behavior and dialogue interactions where gaze between both speakers,
audience members, and objects in the environment ought to be coordinated.
This also includes anticipatory gaze to facilitate conversational, turn-taking,
and otherwise signal the perceptual expectations of a robot.
Other areas of future work include using the integration to enable active
perception, that is, the use of gaze shifts to enable Proactive
information gathering as well as cross modality attentional cueing.
Our section in the NRL has developed
capacities to use auditory cues to capture visual attention and it would be
straightforward to use this integration to develop this in an HRI context.
Finally, in our paper we present one
possible way of integrating DIARC and ARCADIA.
Other possible integration configurations
could also be explored, including using multiple instances
of Arcadia to have dedicated focuses of attention for specific robot modalities.
To wrap up, I wanted to acknowledge
my colleagues and collaborators at both the Naval Research Laboratory
and the Tufts HRI Lab who helped make this work possible.
Likewise, this work was supported and made
possible by awards from the Office of Naval Research.
Thank you for your attention and we look forward to your feedback and questions.