Hello everyone, I am Wenxin, a reseach scientist at Google. Today, I'm going to
introduce HGaze Typing, a text entry interface that combines head gestures and gaze input.
Why We want to combine gaze and head input? Let's first see the advantages
and disadvantages of these two inputs. First of all, gaze input is known for its speed.
Eye movement is fast and allows rapid saccade between fixations.
In addition, eye movement involves few muscles. So gaze input can reduce the risk of fatigue or
cumulative injury caused by long time usage of computing devices. However, the noise of
gaze input can be problematic for fine positioning tasks. There are two source of gaze input noise:
one is the involuntary eye jitters and the other is the noise from eye trackers.
Gaze input also faces perception and control dilemma. That is, eyes are perceptual organs
rather than control organs. When a user is performing a control task, such as dwell-time
selection, the user cannot perform visual tasks, such as checking the entered text.
As for head input, stability is one of its advantages. Also, when performing commands
with head-tracking systems, users can simultaneously receive visual information
or feedback with their eyes, and perform actions with their head movements. However,
head input is slow. And when using head-tracking system for long, users will feel fatigue.
Based on the characteristics of head and gaze inputs,
we derived three design principles: (1) using head gestures to perform tasks requiring high accuracy,
(2) using head gestures for commands associated with visual activities, and (3) using gaze inputs
for tasks that can benefit from fast movements and only require a course level of accuracy.
Let's apply the design principles to gaze-path-based text entry interface.
This kind of interface consists of five tasks: (1) initiating or finishing a word path by selecting
the first letter and last letter, and that requires accuracy, we will use head gesture;
(2) connecting the middle letters of the desired word to generate path and that only requires
course level of accuracy and will benefit from speed, so we will use gaze input; (3) canceling
a path when it started involuntarily and that will require accuracy, we will use head input;
(4) choosing a word from a candidate list and that usually associates with visual activities,
so we will use head gestures, and (5) deleting an unwanted word and that also
associates with visual activities and also requires accuracy, we will use head gestures.
So what kind of head gestures we will use? A nodding gesture is faster than the other head
gestures and is considered the best head gesture for making a selection, so we assign the nodding
gestures to start and end a gaze path task. We use a head shake, defined as rotating the head to the
left or right to a word deletion. Word path cancelling is a similar concept as deletion,
so it's also assigned with the shaking gesture. As for the selection of candidates,
we use tilting gestures. The direction of a tilting provides easy navigation on the word list
Let's take a look at the HGaze Typing interface. Assuming that we want to type the word "gaze".
We will first look at the letter "g", and then the red dot demonstrates the fixation.
The user can perform a nod to select the letter "g".
The boundary disappears, and the word path starts. The user can quickly scan through "g-a-z-e" and
perform a nod at the last letter "e". A candidate list will show above the key,
and "gaze" is entered in the text box. If the user is okay with "gaze", then the user can perform the
next word entry. If not and the user finds the desired word in the candidate list, the user can
tilt his/her head to left or right to select. Say "great", "give", and that will show in the
text box. And if the user didn't find the desired word, the user can use shaking to delete the word.
For system design, we have two algorithms - fixation estimation and head gesture
recognition - to process the gaze and head movement data. And we design four status
in the system, and the most important part is the down gaze lock and gaze path restoration.
When the system detects a downward head movement, a gaze lock is activated to prevent the gaze shift
due to a potential nod. Once a nodding gesture is completed, the gaze position information is
unlocked and restored for gaze path classification. We propose the gaze
lock mechanism and the gaze pass restoration algorithm to minimize effects of gaze shift
for head movements. We use Fréchet score to compute the word candidate list. When we
detect nodding, shaking, or tilting gestures, and then the corresponding command will be activated.
We conducted a user study with 10 participants to compare HGaze Typing with a baseline
dwell-time keyboard. Tobii eye tracker 4c was used to collect gaze and head input.
Each participant typed about 20 phrases every day, and the experiment spans in three days.
Overall, the mean text entry rate using HGaze Typing was higher than using the
dwell- time keyboard. In the last three sessions, a significant effect of interface was found, with
an average typing speed of 11.22 wpm for HGaze Typing and 9.53 wpm for the dwell-time keyboard.
The mean number of deletes and cancels when using HGaze Typing is shown here.
The average number of deletes reduced from 1.75 in session 1 to 0.57 in session 9,
and the number of cancels reduced from 1.05 to 0.6.
HGaze Typing has a higher perceived performance score and is preferred by participants.
Participants like the concept of using various head gestures, especially the concept of using
the tilting gestures to select words. One participant indicated that the "nodding-glancing
through the keys-nodding pattern" creates a rhythm for this bi-modal text entry interface.
To conclude, HGaze Typing achieves a better text entry rate compared to dwell-time typing
and is more comfortable to use. Head gestures provides natural and efficient command activations
and free the eyes for other visual activities. HGaze Typing manages the processing of the
gaze and head inputs in one system without interference. HGaze Typing design reduces
the virtual keyboard area by putting the candidate list between the keys. And the
design model that uses gaze path plus text entry command can be extended to other devices