Welcome everybody and thank you for your
interest in our paper. My name is
Jonathan Liebers, I am a research
assistant at the hci group of the
University of Duisburg-Essen and on
behalf of my co-authors Patrick Horn
Christian Burschik, Uwe Gruenefeld and
Stefan Schneegass I will be presenting
our work on Using Gaze Behavior and Head
Orientation for Implicit Identification
in Virtual Reality here at this year's
VRST to you.
user identification measures on
state-of-the-art head-mounted displays
for virtual reality is a recent research
topic and required to for example
personalize the device
through personalization we can for
example set user preferences within an
application or adapt the difficulty of
games
here you can see the currently widely
employed authentication method of
entering a password in virtual reality
with a handheld controller. It looks
cumbersome as it is does it not? For this
reason we want to explore a gaze-based
behavior biometric that implicitly can
enable authentication in vr
geez is considered to be a highly unique
and individual personal trait often used
for behavioral biometrics not only can
we use the physiological features of the
human eye such as an iris pattern for
authentication but also the behavior of
the eyes itself
the eyes can be tracked with an eye
tracker that is located inside a head
mounted display and they can perform
various movements such as fixations
where the eyes stand still and focus on
one point smooth pursuits where the eyes
move slowly and track a moving object
and saccades where the eyes jump fast
from point to point for example during a
reading activity
we believe that these behaviors are
highly unique and individualistic and
can be used to identify people
in our work we want to exploit these
behaviors and in particular their
associated gaze velocities for user
identification
our core idea is that an immobile
stimulus elicits fixation eye movements
whereas a slowly moving stimulus induces
smooth pursuits once the stimulus
moves faster and faster a smooth pursuit
might not be enough to follow it anymore hence
saccadic eye movements are elicited
from a certain point on it might also be
necessary to support the saccadic eye
movements by performing an additional
head rotation we hypothesize that the
obtained values for the eye movement's
velocity as well as the transition
timings and the speed of the head
rotation are highly individual and in
fact usable for authentication in vr
we designed a user study to investigate
this hypothesis in order to investigate
the differences in gaze behavior between
persons we use two dynamic stimuli each
on their own trajectory in virtual
reality
here indicated in red is the field of
view in vr
the inner ellipse represents trajectory
one for the first stimulus that stays
within these bounds at all times whereas
the outer ellipse exceeds these bounds
both stimuli increase their speed over
time but as trajectory 2 exceeds the
field of view a head rotation must be
induced to follow it continuously with
the eyes
to evaluate our work we conducted a user
study in our lab and invited 12
participants where two identified as
female antennas male
the age span ranged from 21 to 28 years
with a median of 25 years
unfortunately during the study one
participant encountered cyber sickness so
we stopped the procedure there and
remained with 11 complete data
recordings
our study followed a within subject
repeated measures design where we
recorded three repetitions per stimulus
to do so we have utilized an htc vive
head mounted display and the eye tracker
add-on for said device that is provided
by pupillabs
at first we want to gain insights into
the data and understand if the seeked
individuality within the data exists
we do so by employing an explainable
machine learning based on k-nearest
neighbors at first
in the second part we will use deep
learning to maximize the accuracy
to evaluate our concept we first collect
the participants data by setting the eye
tracker to a resolution of 400 times 400
pixels at 120 hertz
we furthermore perform the vendor's
default 16 point 3d calibration routine
after collecting the data we apply a
confidence filter of 0.4 to the data
this filter replaces all recorded data
points with not a number of markers that
indicate that the eye tracker was not
confident at the time of recording as
for example it is during blinks
next we classify the stream of data into
segments where saccadic movements
fixations of smooth pursuits
occurred
after we obtained the segments that were
classified as fixations smooth pursuits
or saccadic eye movements we create
feature vectors by applying summary
statistics
by especially focusing on the mean peak
velocity
for stimulus 1 we obtain a 3-dimensional
feature vector whereas we include the
head rotation for stimulus 2 and obtain
a 4-dimensional feature vector
at last we classify these feature
vectors in an 11-fold cross-validation
where we designate one of the 11
participants in turn as an unknown
person to be rejected by the system
we utilize a k-nearest neighbour's
algorithm that makes use of a distance
threshold and sets the parameter k to 1.
for stimulus 1 we obtain accuracies
between 36 percent and 55 for the worst
and best case respectively in total we
find a mean accuracy of 45 percent
to the right you can find a plotted
version of the knn where the colors
denote the participants data on a log
scale
for visibility we only depict the smooth
pursuit and circade peak velocities and
by the color clusters we can see that
there is some inherent similarity within
the data
alike for stimulus 2 where we include
the head rotation we find an overall
increase where the best case equals 82
percent and the worst case equals 73
in total we find a mean accuracy of 75
percent
since we now saw that there is inherent
similarity in the data we try to
maximize the identification accuracy we
do so by utilizing deep learning which
is known for being capable of performing
the required feature engineering on its
own
in the first step in terms of resolution
sample rate and calibration all values
are equal to the previously presented
machine learning evaluation
but since anomaly detection for deep
learning is a current research topic and
the rejection of unknown data samples is
as well we focus on a pure
identification scenario instead where we do not
designate an identity to be rejected
here
in the second step we again apply a
confidence filter of 0.4
here we do not set values to not a
number but perform an interpolation
instead as the deep learning models
cannot work with not a number of markers
in the data
moreover we equalize the different
sampling rates of our data by up
sampling the head rotation data to match
the gaze data to fit them into one data
frame with no dilation on the time axis
in the third step we apply a clipping
and min max normalization to fit all
values into the range of 0 to 1.
moreover we apply a prepadding and make
use of the sliding window technique
where we set 300 as the window size
which corresponds to 2.5 seconds and 150
to the stride so that we have an overlap
of 50 percent
at last we perform a three-fold
cross-validation where we use one
recorded sample per turn as the
validation sample
we moreover utilized ten different deep
learning model architectures as proposed
by Fawaz et al. in their survey paper on
deep learning for time series
classification
as a result we obtain very high accuracy
values where the encoder model stands
out by a mean accuracy of 97 for
stimulus one and one hundred percent for
stimulus too
these values are to be understood after
the application of the majority ward
that operates on the prediction of the
single slices created from the sliding
window technique
here we can see the overall determined
accuracies particularly the encoder
model appears to be a good fit amongst
others as the accuracy is relatively
high across the board yet it needs to be
noted that the other models proposed
by Fawaz et al. at all that are not listed in
this table did not converge
so what can we conclude in the end first
we see that gaze can be used to
implicitly identify users
we also find that head rotation overall
improves identification accuracy
we believe that these findings can be
used for implicit identification where a
user just follows a stimulus in vr with
their eyes that ideally includes a head
rotation this could be a ball in a
spectated tennis or football game or a
virtual airplane in the sky
thank you for your attention