- Yeah, good morning, everyone.
I'm Keita Higuchi from the University of Tokyo in Japan.
Today, I will talk about a study
of remote collaboration using human gaze information.
First, I show a research background.
In this study we focus on asymmetric remote collaboration.
This form of remote collaboration
has two different roles of users.
This framework is able to support local workers
by remote collaborators' instructions.
A system captures the workspace
and sends each to the remote collaborator.
The remote collaborator gives some instructions
to the local worker.
This kind of system is designed
for seamlessly connecting between users.
This remote collaboration framework
is an important topic in HCI
and it goes back to early 90s.
Kuzuoka designed a gesture instruction framework.
It transmits collaborators' hand gestures
to the local worker via RHMD.
Kurata introduce laser pointing
to point out objects in specific, in physical workspaces.
As a systems, used annotations made by collaborators' hands.
The target focus of these studies
was explicit instruction via collaborators' hands.
In this work, we proposed visualizing eye fixations
for novel instruction cues,
under this kind of remote collaboration framework.
This image contains or displays
to show a workspace view to the collaborator.
We set an eye tracker under the display.
The system tracks collaborator's eye movement
and maps gaze position to the workspace.
We investigated visualizing eye fixations
in two types of setting, to generalize its effects.
Generally, there are two main camera positions
in the remote collaboration framework.
The first setting is a fixed point of view camera,
that provides a stable view to the collaborator.
The other is using a wearable camera
that can move the camera position by worker's movement.
We need to study both cases, because they have
different advantages in supporting remote collaboration.
In this study, we introduced two types of scenarios
of remote collaboration by using fixed device
and wearable device settings.
In each scenario, we investigate
how visualizing eye fixation helps remote collaboration
by conducting a three part user study
We explain important findings from the experiment
about transmitting explicit instructions
and implicit intentions via eye fixations.
This video shows our fixed device setting.
The system visualizes collaborators'
gaze position on their trails.
When the collaborator points out blocks
to instruct an employee during the task,
the system also displays hand gestures
of the collaborator on the workspace.
The worker and the collaborator can also
talk to each other by audio.
This video shows our wearable device setting.
The collaborator can see the view of the workspace
from the worker's point of view.
The collaborator looks at the target object
in the worker's view.
The collaborator's eye position is shown
to the worker by the HMD.
This slide shows our system configuration
of the fixed device setting.
The collaborator side has a display and eye tracker,
the worker side has a camera and projector.
We perform projector-camera calibration
before using the system.
In this video we can see how the collaborator's
eye position and its trails are
displayed on worker's workspace.
The green circle represents collaborator's position,
and the green line shows its trial.
This slide shows the system configuration
of wearable device setting.
On regular fixed setting, the worker
has a wearable camera on the head-mounted display.
The wearable camera's view changes according
to the worker's head movement.
Other important difference is the worker's HMD
has very limited field of view.
We use optical see-through HMD.
This can directly provide information from worker's eyes.
As we can see, there's a picture,
the HMD's field of view, illustrated with the yellow square,
is much smaller than the
worker's wearable camera's field of view.
Therefore, the collaborator's eye position cannot be shown
to the worker unless it is inside the yellow square.
To overcome this issue, we designed
an indicator of gaze direction and distance,
with arc position and length.
The green circle shows exact gaze position.
The collaborator's hands are displayed
on the worker's workspace.
This is done by using a polarizing filter,
attached to the camera on the worker's side.
The polarizing filter blocks all eyes from the display,
while the camera can see the collaborator's hand.
The captured hand regions are
projected on the worker's workspace.
We designed a three part user study
to investigate the effectiveness
of visualizing eye fixations.
The first study is about an assembling task.
The collaborator gives instructions
how to build a predefined object.
We saw qualitative and quantitative results
in the fixed device setting.
The second study is about a task to study
a pointing ability in the wearable device setting.
This is a very simple task.
The final study involves a more practical task,
more specifically, arranging a showcase in a room.
In the first study, participants aimed to build
predefined objects using wooden blocks.
First, we showed the target object to the collaborator.
The collaborator then gave instructions to the worker
using speech and other instruction modalities.
A participant did a task under two conditions.
First condition, they used hand gesture and speech.
For the other condition,
they are also able to use their gaze for instructions.
We evaluated the task completion time,
and we also interviewed about participants' experiences.
Participants made two person groups,
and played both collaborator and worker in a group.
This is the result of task completion time.
The gesture and eye condition made the faster results
than gesture only condition.
The participant used eye fixations for pointing out blocks.
Eye fixation could arrive first pointing,
because eye movement is very fast.
We observe a use of hand gesture.
It is used for describing object manipulation,
such as how to attach a block.
Some participants mentioned they could
predict a target of next instruction blocks
from the eye movement on the workspace.
The second study is an objective study to investigate
the pointing ability in the wearable device setting.
The participants paired with the remote collaborators
and pointed out a single block to identify the object.
This task was actually not easy,
because workers could not see instructions
in the narrow view field of the HMD.
We compared the two conditions.
In the first condition, the participant
used the hand gesture,
and in the other condition,
they could use their gaze to identify objects.
We eliminated a number of mistakes on task completion time.
This is a task procedure.
Before the task, we showed the participants
where the target block is located on.
The participants then pointed out the
target block to the worker.
The worker tried to find the target block from 15 blocks.
When the collaborator finds the target, the collaborator
judges the block whether it is the target or no.
If incorrect, the worker again tried
to find the target block from other blocks.
Each participant took nine sessions of the task.
This slide shows the results of study two.
In the result, eye fixations could reduce the number
of mistakes in nine sessions.
Eye fixation also marginally made
faster results than gesture condition.
The results show accurate and fast ability
of eye fixation in wearable device setting.
The third study is designed to observe
participants' qualitative experiences.
This is a practical task.
The participants arrange a showcase for a trip fair.
They first decided the design of the showcase,
then started decorating the showcase
using the working desk and toys.
The workspace had a shelf to contain toys,
and a working desk and showcase space.
We employ the gesture and eye condition.
We observe how the participants use
the instruction modalities during the session.
This slide briefly shows the results of study three.
The pair of participants succeed in making their showcase.
The participants used eye fixations
for pointing out objects or places.
When the collaborator gave
pointing instruction via eye fixations,
they used their speech like "can you take it?".
Therefore, the workers easily distinguished
collaborator's instructions from eye fixations.
The worker mentioned that they could know
where the collaborator is interested in,
because the collaborator's eye position
are always shown on the HMD.
Okay, finally, we reported important effects
of visualizing eye fixations from the studies.
The first finding is pointing ability
of visualizing eye fixations.
In the first study, we observed
faster object identification.
Visualizing gaze trails enabled
multiple pointing at the time.
In the wearable device setting,
the collaborators achieved accurate pointing
in the dynamic scene, captured by a head-mounted camera.
The second finding is different roles
between instruction modalities.
Eye fixations are mainly used
for pointing out specific objects.
Hand gestures transmitted detail of instructions,
such a manipulation.
In the third finding, we observed a combination
of eye fixations and speech.
When the collaborator gave speech instructions
with eye fixations.
The worker could easily distinguish instruction
of eye fixations from other natural eye movements.
The final finding is related to the previous one.
The worker can predict some implicit intentions
from natural eye movement of the collaborator,
when the collaborator did not say anything.
In the fixed device setting,
the workers said they could estimate next instruction.
In the wearable device setting,
they roughly know where the collaborator is looking at.
This is the summary slide.
We designed visualizing eye fixations
for remote collaboration.
We performed three user studies with two settings.
And finally, we reported major effects
of the visualizing eye fixations.
Here, I'd like to say a take home message.
Visualizing eye fixations supports natural communication
in remote collaboration.
Okay, this is the final slide, sorry.
I'd like to conclude this presentation.
Our top, high level research question is,
"can eye help you?"
Okay, from this study, finally,
we can now conclude, "yes, eye can!"
(laughter)
Thank you very much.
(audience applause)
- [Woman] So, a short question from me,
you did mention some differences in how
the hands were used...
- The hands were used for describing manipulation
of where the block is located on.
And the eyes were mainly used for identifying objects.
- Okay, but...
- Sorry
- Yeah, that's fine.
Okay, so, if there is no questions from the audience,
then I think we'll let the next speaker up.
- Thank you very much.