Head movement is a compelling form of input modality because it affords
hand-free and non-verbal interaction.
However, not all type of head movement are the same.
We turn our head to see what's around us, and we use head
gesture to express ourselves.
There are fundamental differences between the type of movement used to
direct visual attention and those used as gestural expressions, we refer to
them as Head-Gaze and Head Gesture.
They're both of utility, but differ in affordances.
Hi, my name is James, a PhD student at Lancaster University.
If you'd allow me, I'd like to share with you how we classify these type of
head movements using machine learning.
And how by treating them as distinct modes of input allows for more seamless
interaction with simpler head movement.
This work is done in collaboration with my colleagues Joshua, Ludwig, Anam, Per
and Hans, with contribution from Dominic.
But first, let's play a game to experience how head movement
allows for hand-free interaction.
There's a list of animal cars in front of me.
Avians and mammals.
I can swipe them to sort them into categories.
Avians to the left, mammals to the right.
Hey, I recognize this dog.
That's my friend Dominic's dog.
I want to look at it.
Oh no, I didn't mean to swipe it.
I turned my head only to look at it.
Now I made a mistake.
This is annoying.
If only we can tell when the head movement is a gesture
and when it is a part of gaze.
A lot of times gestures have to be designed to be exaggerated
movement to avoid accidental activation or avoided altogether.
If we can distinguish head movements that are gaze-driven and head movements
that are expressions of the head in its own right, we can treat them as
distinct modes of input and allow their use in tandem, we call the type
of head movement that is gaze-driven,
Head-Gaze.
Head-Gaze supports the eyes in directing visual attention to objects
that are not already right in front of us, and to keep the eyes within
a comfortable eye-in-head position.
In contrast, when we move our head independently of gaze to describe a
gesture, then it is the head that leads the interaction and the visual system
that adapts making it possible to use head movement as means of expression,
without disrupting vision, we call this type of head movement, Head Gesture.
Head-Gaze is the type of head movement driven by gaze to support
us directing visual attention.
While Head Gesture is independent of gaze and used as gestures, the
classification of head movement as Head-Gaze or Head Gesture not only
reduces false activation, it allows us to leverage the two types of head
movement, in more improved and novel ways.
For example, if I have two displays, each contains a deck of cards.
The one I'm attending to is the main display.
I can use Head Gesture to swipe cards in it, and when I look over to the other
display, a Head-Gaze is detected instead of falsely activating a swipe, it centers
the now-attended display, which allows me to use Head Gesture to swipe cards.
I can use Head-Gaze to switch between multiple displays and
Head Gesture to complete actions only in the attended display.
Therefore, the classification of Head-Gaze and Head Gesture allows multiple
head-based systems to coexist without accidentally triggering each other.
Another benefit could be the seamless switching of interaction
modes in the same application depending on the user's current.
For example, switching between gaze-pointing and head-pointing
for ergonomics and the precise control of the cursor.
This is an adaptation of the BimodalGaze technique by Sidenmark and colleagues
who inspired us in this work.
Initially, the cursor shown by the white square is controlled by gaze due to
tracking inaccuracy, the cursor is offset from the true gaze shown by the red dot.
When the cursor lands on the card, its information is displayed.
Now I want to look at the Shih Tzu dog, but the eye tracking
is not accurate enough.
A Head Gesture switches to head-pointing to allow precise control of the cursor and
Head-Gaze switches back to gaze-pointing.
Let's take a closer look at Head-Gaze and Head Gesture and what challenges
we need to overcome to classify them.
When I look at a target, my eyes move first and acquires the target.
Head is slower and most of the head movement happens while
gaze is already on target.
As the head catches up, the eyes compensate for the head movement to
fixate on the target, this is observed as opposite movement of eyes and head known
as the vestibulo-ocular reflex or VOR.
You can see this on the plot here with the blue eye-in-head angle and the orange
head angle move in opposite directions.
The head movement here helps us direct the visual attention to look
at the target, so it's a Head-Gaze.
Now I use Head Gesture to move the ball into the black hole.
The same relative movement of head and eye is observed during Head
Gesture where the eyes compensate head movement to stabilize vision.
Making it difficult to distinguish gestural from gaze-driven behavior.
We explore a machine learning solution and design a task to collect Head-Gaze
and Head Gesture data while automatically labeling the movement phases for training
. First participants gaze shift to acquire the target with associated head movement.
Then they perform Head Gesture to direct the ball into a hole.
We vary the sensitivity of how head movement is mapped to the movement of the
ball by varying the control-display gain.
In order to draw out head movement of different amplitudes and the velocities.
We employ the common features for eye- and head-based classifications, including
shape-based, noise-based, spectral-based, correlation, and timing-based features.
To capture the temporal history of participant behaviour,
we included the feature vector from the previous one second,
sampled at regular intervals.
We used an extreme gradient boost or XGBoost classifier
to classify head movement into either Head-Gaze or Head Gesture.
We call our system HeadBoost.
Its person-independent offline performance achieved an F1-score of 0.89.
HeadBoost is able to classify very small Head Gesture and detect the onset of Head
Gesture earlier than a heuristic baseline, we calculated the feature importance
and report that no single feature is solely responsible for classification.
Rather, a combination of head-based and temporal features contributed to
predicting the type of head movement.
And finally, to demonstrate HeadBoost's real-time utility and
practicality, we implemented the card game application and showed that the
classification of Head-Gaze and Head Gesture enables designers to create
a more seamless and natural way of interaction using simple head movements
while avoiding false activations.
In this work, we make three main contributions.
We introduce the fundamental differences between head movement driven by gaze,
Head-Gaze, and those independent of gaze , Head Gesture to make case for
their use as distinct modes of input.
The stimulus and task design is able to separate Head-Gaze and Head Gesture
for automatic labeling of head movement.
We developed HeadBoost, a novel classifier to classify head movement
into Head-Gaze and Head Gesture.
It achieved a person-independent F1-score of 0.89.
We further demonstrate its feasibility and practicality with three applications.
We elaborate more on these ideas in the paper.
If you're interested, we would love to hear from you.
thank you for listening.