- [Announcer] After lunch.
So I see people are still
coming because it's quite a challenge to get in that speed
in that time lunch, but great that you are here.
One announcement up front, we have in this session
only three papers.
If you rely on the paper program, that's outdated.
The online program is already updated.
The second talk, Smooth Moves, has been moved to
tomorrow morning to the nine o'clock session because
the authors and presenters are still on the plane,
so there was a problem with the flight.
So we will move the third and fourth talk
20 minutes to the front, the others (mumbles).
And so we have three talks and the first talk, we have
Mohamed presenting Eye Scout, Active Eye Tracking
for Precision and Movement Independent Gaze Interaction
with Large Public Displays, and this is joined with from
Ludwig Maximillian Universities in Munich,
and the Max Blank Institute.
And my name is Albrecht Shmid-in-fam.
I just moved to that institution a few weeks ago,
so I'm quite curious.
- Thanks for that introduction, Albrecht.
So, high everyone, my name is Mohamed, I am a PhD student
at the University of Munich, it's a pleasure to present
this work today.
So I'm presenting on behalf of the co-authors, Axel Hoesl,
Alexander Klimczak, Martin Reiss, and my supervisors,
Florian Alt, who is here and Andreas Bulling
from Max Blank Institute.
Alright, so, we've already had eye tracking and gaze
interaction for a pretty long time, like 30 years or so.
However, most of the research that has been done in eye
tracking has been focused on desktop scenarios or cases
where you have small displays.
So in these scenarios you're expecting most of the time
that the user is not moving a lot, the user is stationary
most of the time and interacting for a long period of time.
However, in this project here,
we focus on large public displays.
So, actually why is this different?
Now, if you imagine that you have a large display
and you want to use one of these commercial eye trackers
or just a regular camera to do eye tracking
or gaze interaction, the first problem you come across
is that people could be approaching from different
positions around the display.
So, it's not like there is a fixed position that is
intuitive for people to go there.
So you have to either guide people to stand at the certain
position where the eye tracker could detect their faces,
or you need to somehow get them to go there.
And also you have cases where people would be interacting
with large displays while walking.
So the question is, how can you enable eye tracking
and gaze interaction for these cases where you have
large displays where you need to be flexible to different
positions, and flexible to the users movement?
So, as I said, the usual ways to ask people to go to
the eye tracker, so you place the eye tracker somewhere,
ask people to go there, and instead of doing that,
we thought that maybe we should
move the eye tracker to the user.
So, we did this through our system, which we call EyeScout.
So basically, we mounted an eye tracker on the rail system,
and we use a Kinect on the other side of the room to
detect where the user is, and we accordingly move the
eye tracker to follow the user as he interacts while
moving, and also to simply place the eye tracker wherever
the user approaches.
So we focus on two scenarios here: a scenario where a user
walks to a random position, then interacts, and a scenario
where the user walks and interacts while walking.
So, why would this be useful?
You can think of the displays
that we often find in airports.
You know, we've got in these airports these moving walkways
or like horizontal escalators, and you've got displays
on your sides usually, advertisements or digital displays.
And if you want to track the users eyes
as the user stands on these moving walkways, you can
actually use a similar system where the eye tracker
moves along the moving walkway.
Also, if you go to...
So this is a picture of Quebec City,
so, there is this (speaks in foreign language),
my French sucks, from which you can see the city,
and you can imagine that you can have an eye tracker
at such an observation tower.
So, simply, you walk to a position in the observation
tower where you look at the landmark, and the eye tracker
would come and the system would tell you what you're
looking at and describe the scene for you.
We are also starting to see more of these cylindrical
displays where people could interact with the cylindrical
display from different positions around it.
And you can also imagine having a similar system surrounding
the display and moving the eye tracker depending on
the users' position.
So, actually, why do we work on public displays?
I'll start talking about how we came into this idea.
So, we are starting to see a higher adoption of large,
public displays, so, not just in research, but also in
industry and research.
So, for example, you've got this big display at NASA.
As I said earlier, although we have eye tracking on such
public displays, the idea of allowing gaze interaction on
large displays hasn't been explored a lot before.
And the reason is that you've got these special situations
where people could be interacting from different positions
and while walking.
So I said earlier, people usually try to guide people to the
so-called sweet spot, so that's the position from which you
can interact with the display.
So if you have an eye tracker mounted at the display,
usually you want the user to stand right in front of it.
So one of the ways was to let us present it in Whist
two years ago, is to guide people through showing
queues on the screen to implicitly guide them to stand
in the right position.
And the other way's to show the users instructions on
where to stand and how to align their face.
However, although this is okay for cases where you have
a smaller display, because it's actually intuitive to
go and stand in the center of a display, with larger
displays it could obstruct the interaction process and
could make the interaction not smooth enough.
So the way we built the system is that we used the
rail system which carries a carriage.
The carriage carries the eye tracker.
And we use a Kinect on the other side of the room to
detect where the user is and accordingly place
the eye tracker.
Of course, the Kinect could be replaced with any kind
of motion sensing device.
I'll speak in more details now about how we actually
implemented this system.
So as I said, we support two interaction modes.
So one of them is walk, then interact.
So you walk to a random position, and then interact.
And of of them is walk and interact, so that just means
actually interacting while moving.
Now the system looks as follows:
It's like a top view, so we have a body tracker, which
in our case is a Kinect.
The eye tracker is mounted on a carriage, and the following
process takes place.
So first, once we detect a body, there's a software
component which computes where the user is and accordingly,
how much the carriage should move.
It communicates this information to a microcontroller, which
in turn, instructs the rail system to move
the carriage to the user so that the eye tracker
will detect the eyes.
As long as the users eyes are detected,
the system stays in place.
If the eyes are no longer detected, we go through this
whole process again.
And this also allows people to interact while
they are moving, this same concept.
Now to build this, we used
a carriage and a rail,
and we drilled some holes into the rail to get the wires
to go through.
We 3D printed this holder, and we drilled a hole at the
bottom to mount it on the carriage.
If you're interested in the detailed
technical implementation, I'll be happy to share with
you a technical report with more details,
so just get in touch with me,
and I'll be happy to share that.
We used a motor and stepping driver so the stepping driver
converts the digital signals to something that the motor
could understand.
We also used a Arduino.
Of course, all of these parts are upgradable, so you
could always use a different,
a more advanced, or a faster motor.
You can use a different controller other than the Arduino.
We connected everything and we also had these safety
measures so we had some switches at the sides of the rail
to make sure that the carriage does not
walk out of the rail.
Then we conducted a user study in order to see how people
really interact with their eyes while they are moving.
So we evaluated the two scenarios that I spoke about.
In one scenario, the user had to walk to a random position
in front of the display, and they would have to then
select something with their eyes.
So we showed three moving objects, two gray objects and one
red object.
The user had to select the red one, and if they select
any of the gray ones it would be counted as an error.
And in the other scenario of walk and interact, they had
to do the same thing while walking along the display.
So we used the pursuits method to interact via eye gaze.
So the pursuits method basically, it's an eye tracking
or a gaze interaction technique where you can select
objects, moving objects with your eyes.
The nice thing about it is that it doesn't require
callibrating the eye tracker, so basically you show the user
moving objects, and then you compare the movements of the
users eyes to the movements of these objects to know
which object the user is looking at.
And, yeah, as I said, the last thing is that it doesn't
require calibration and it's also brought us to
noisy gaze data.
So in the first scenario, they had to do three selections,
and the second scenario, they had to do as much selections
as possible at three different speeds.
So the user had to walk at different speeds, and this was
our way to find if people are okay with faster speeds
of moving, and interacting or selecting via gaze
while moving.
So we recruited 26 participants.
11 of them were females and we measured the error rate
and the selection time.
So when looking at the error rate, we found that actually
the error rates do not change much
throughout the conditions.
So we actually found that the higher the speed,
the more likely people will do some errors, however,
the difference is not significantly different.
So, this means that people are more likely to do more
errors when they are walking much faster.
And for the selection time, we didn't find much
differences also, except that, in general, selecting
objects while walking takes more time compared to
selecting objects while stationary.
And the reason is that since we are using this pursuits
method, the way it works is that you try to achieve
a certain correlation between the eye movements
and the movements of the objects shown on the screen.
And to reach this threshold, you usually need more time
when you are walking, because when you are walking,
your head bobs up and down and this distorts
the gazing habit.
We also collected subjective feedback from the participants,
so we asked them about the perceived
precision of the selection, and also the perceived easiness
of the selection.
So for precision we didn't find a lot of differences, but
for easiness, we found that people preferred selections
while stationary and while moving on lower speeds
compared to faster speeds.
So as I said, the system could be actually upgraded
in many ways.
So one way is that you can actually conceal the rail.
So you can imagine having a display and the rail actually
behind a wall or behind a frame with a small hole that shows
so that the eye tracker could see the eyes.
So you could actually conceal the rail, so that it wouldn't
also distract the user.
You can also upgrade the motor to support faster speeds
or slower speeds if you like.
You can use different motion sensors, so in our case,
we used the Kinect one.
You could also track the entire room, so you could
put the Kinect on different positions in the room.
Also, in our case, when we have participants that are
taller or shorter, we would ask them to either step
back, in case they are taller, so that the eye tracker
would see their eyes, and if they were shorter, they would
have to approach the display a bit.
And we showed cues, texts, prompts on the screen to
guide them to do that.
But a future system could also pan and tilt the eye tracker
in order to adapt to these situations.
And of course, although we used Pursuits, we could also
use different eye tracking techniques.
So we could use gestures, for example, so we all recommend
whenever there is so much dynamics while interacting,
it's better to use something that is robust to calibration
errors, so for example using gestures or using pursuits
would be a good approach.
There are also several methods introduced in research,
for example, there was a paper at Kai called Eye Grip, where
they used an eye movement type called optokinetic nystagmus,
and that's actually very good,
very robust against calibration errors as well.
And actually our limitation of our system is that we
support a single user at the moment, and that's something
for future work that we could think about ways to support
multiple users.
So the main take home messages from here is that first,
so the method that we used here
is called Active Eye Tracking, so actually moving the eye
tracker depending on the user, or adjusting the system
depending on the users position as a form of
active eye tracking.
So we think it's a really promising direction for gaze
interaction with large displays.
So if we compare the results of our study to previous work
where people were first guided to position themselves
in front of the eye tracker and then took the time to
align their faces and so-on, we would find that
kickstarting the gaze interaction process using EyeScout is
62-87% faster, so it's actually a huge improvement.
And we also, using the system, we were able to extend the
sweet spot to a sweet line.
So the sweet spot when interacting with a display is the
best spot from which you can interact with the display.
So in that case, if you have a stationary eye tracker in
front of a display, the sweet spot would be the point in
front of the eye tracker.
However, using EyeScout, then we have some sort of a sweet
line, so there is a bigger range of area where you can
interact from.
And finally, we found that gaze interaction while moving
is feasible, and this was the first time to have
eye tracking, or gaze interaction while you are moving.
However, it's also less accurate and requires more time
compared to when the user is stationary.
So yeah, that's all.
I'm looking forward to your questions.
(applauding)
- If you have a question, please use the microphone and
state who you are.
- [Andy] Hi, Andy Wilson, from Microsoft.
Thanks for the talk.
It occurs to me that you could have calibrated the rail
against the geometry of the display, the physical display,
and then when you got an eye gaze point from your
eye tracker, you could then relate that directly to the
geometry of your display.
- Okay?
- [Andy] Right, and you did not do that right,
you didn't do that.
- So, you mean to locate where exactly on the screen the
user is looking, right?
- [Andy] Yes, well you need to know precisely where the
eye tracker is, it's probably in 3D.
- Yeah, it's possible to do that.
We didn't do that here, but we used a calibration-free
technique, so we really don't need to know where exactly
the user is looking on the screen, we just compare
the eye movements.
- [Andy] I get that, but I'm curious, I mean, I think that's
a nice observation, but it also would have been nice to
assess the baseline accuracy if you had just followed
through with that and actually got the 3D geometry
and sort of done sort of the, in my opinion, sort of
the obvious thing.
- Yeah, I agree, that's something interesting to do.
So, we haven't done that, but that would be a good direction
for future work.
- [Andy] Okay, thank you.
- [Steve] Hi, Steve (mumbles), Columbia University.
So, you have a really interesting trade off here between
one eye tracker and a rail that needs to not nip people's
fingers and motors that need to pull things back and forth
along it, and the fact that with one single one
going back and forth, if two people want to look
then unless they're walking
together, they can't look at the same time.
And, it would seem to me that eye trackers are just
cameras and diodes and cameras are getting cheaper and
cheaper, have you thought about the alternative of just
put a bunch of just really inexpensive eye trackers into
something which could be a rail like that, which wouldn't
have any moving parts, and then you could have multiple
people looking rather than just a single person, without
the moving parts?
- Yeah, that's a very good idea and, so definitely having
multiple eye trackers would be also a good option,
however, you would have to take care of like, handing over
from one eye tracker to the other, which is also a
complicated problem on its own.
But definitely it's also another approach
to achieve the same thing.
- [Steve] Thank you.
- [Woman] Hello, I am (mumbles) from McGill, and my question
is, you stated the application of eye tracker in the
airport, where a person is walking and
it's tracking the path.
What if there are so many people walking in front of the
eye tracker, so how is this applicable at that time?
- So, at the moment we support a single user, so this means
that if there are several people, it's difficult to detect
their eye gaze using the current system,
but what you could do, I mean,
we use the commercial eye tracker,
which is built for a single user, however, you can use
a regular camera and then you can build your own
eye tracking approach and then you can
support multiple users.
So, this would be an option and of course, you could
also consider having multiple rails
to support multiple users and kind of have several
carriages moving back and forth to support
as much users as possible, so this could also
be a direction for future work.