We are delighted to present our research findings
and share insights on improving
and Analyzing Sketchy High-Fidelity Free-Eye Drawing,
reported by the first author, Lida Huang.
Free-eye drawing refers
to using only the eyes to create drawings,
thereby replacing using hands in
conducting free-hand drawing.
This paper aims
to improve free-eye drawing by
resolving the remaining problems
associated with the previous studies
and analyzing the usability of these gaze
control methods in different aspects.
The main contributions of this paper are forfold:
A fully-functional free-eye drawing user
interface tailored for gaze interaction
that enables high-fidelity digital graphics editing.
A unimodal gaze control drawing manner,
mimicking the natural brush
start-stop action of hand drawing
and a better way to make observations.
A double blink gaze gesture
that was adopted for
free-eye drawing and can be generalized
to gaze interaction.
User studies and system evaluations
offer comparisons with different gaze control methods
and real-world drawing by hand,
facilitating recommendations regarding improving
eye-tracking techniques and gaze interaction.
Today,
the primary gaze interaction
form is that what you look at is what you get.
This method has two features:
the gaze position or trajectories
are immediately output as pixels;
the applications that follow this method
cannot conduct a click-and-drag mouse event.
They can only execute mouse click
events via dwell click,
namely,
''dwell click to draw dwell click to and drawing'';
thus, they can only create straight lines
connecting the brush start point and endpoint.
This type of gaze control
is challenging for free-eye drawing,
as observation requires the eyes
to fixate on the sketches,
whereas controlling the brush
requires the eyes to make saccodes.
Sacods require
step-like changes and ballistic movements,
whereas accurate brush control requires minor adjustments.
To address the above problems,
we proposed a novel gaze-control method
named compass in our previous research,
which emulates click-and-drag
for drawing continuous strokes.
A circular cursor
can trigger the eyes
to make smooth pursuits along its radius,
drawing smooth lines and diverse shapes.
The directional needle
points from the brush point
to indicate the direction of the brush movement.
The traditional command for
starting and ending a drawing is the dwell click,
which means the user
rests the gaze on a point for a prolonged period
to activate or deactivate the brush.
However,
using dwell click
to end a brush stroke is error-prone approach
as it will most likely lead to endpoint tailing
in free-eye drawing.
Considering the challenge described above,
we employ double-eye blinks as an alternative method
to end a brush command quickly.
The application is built with C#
and runs on the Windows operating system
using the Unity engine.
The interface aligns with Fitts’ law of gaze control;
For example,
the buttons are 100 pixels wide times 120 pixels high.
To avoid adjacent interface elements,
the buttons most frequently
used during the drawing process
were placed in the screen's four corners.
These buttons can be reached quickly
and are distanced from the visual attention area,
namely, the canvas,
to minimize the chances of the user
unintentionally activating the buttons.
This application employed a dwell click of 0.5 seconds
on the canvas to start drawing
and a quick double-blink to end the drawing process,
while the command for button selection
remains a dwell click,
namely, 0.5 seconds.
The pan Canvas function
adopted an auto-scroll capability,
allowing the user
to pan the canvas vertically
and horizontally if the user looks at an active region.
The four corner buttons
were designed to remain continuously clicked
after a one-time dwell click
where the gaze stays on the button,
avoiding unproductive mouse activities
and shortening interaction time.
We adapted the algorithm below
to mimic the inherent brush
start-stop actions in hand drawing.
With this algorithm,
the brush will not move
when the user makes visual observations
within the Compass circle.
Therefore,
the user can
pause the brush while conceiving the areas to be drawn
and start drawing easily,
replicating the frequent
start-stop action in real-world drawing.
Furthermore,
the damping coefficient can restrict drawing speed
at a similar speed
to hand drawing speed.
We applied a novel double gaze blink gesture
to exiting drawing,
which has higher accuracy than
a single blink gesture
and higher efficiency than the dwell click.
First,
we examine the eye blink frequencies
for the two gaze control scenarios;
the probability of two continuous
spontaneous blinks occurring in fewer than 2.6 seconds
is significantly low.
Therefore,
a time interval of < 2.6 seconds
for double-eye can distinguish
between an intentional command
and unintentional eye closures.
We also tested the accuracy rate
for stopping drawing via
different time thresholds in our application
via Tobii 5.
Table 1 shows that two consecutive blinks
within 0.4-0.6 seconds is robust,
as the double blink command
conforms to eyelid movements in our environment
and similar eye trackers.
In the final application,
we adopted 0.4 seconds as the default time threshold
for a double-blink click.
In this paper,
a drawing experiment equals (N=45)
was conducted to compare three methods
the real-world drawing with hands, our method
and the previous method.
The three screen recordings
displayed here represent the results
drawn via the three experimental methods,
each participant was assigned a sample sketch
and was asked to replicate the sample
using the three drawing methods,
and try to produce as similar as sketch as possible
through each method.
The drawings via the different method show
that are application leverages the design of Compass
to enable smooth pursuit
and easy drawing with the eyes.
The fidelity is improved significantly compared
to the previous gaze control drawing method
and resembles the results obtained
from hand drawing results.
The previous gaze control drawing method
in the experiment represented most of the gaze
control studies were based on the form of
“dwell click to draw, dwell click to end drawing,”
where the gaze
could only determine the start and end points,
thereby limiting the user
to creating a single line segment for interaction,
reducing efficiency.
The AOI results showed significant differences
between each pair of groups among the three methods.
It indicates the efficiency
and engagement of gaze interaction.
The AOI of our approach was significantly higher
than the previous one.
There are several reasons for the improvements.
For example,
by reducing the number of
dwell clicks through both the UI
and the double blink gaze gesture,
the attention
and productivity of the drawing process are enhanced.
The improved compass and brush dynamics
also minimize distractions from weak interactions,
enabling the user to concentrate on the drawing task.
6.2.3 Ease of Use.
The raw NASA TLX scores indicate
the participants experienced
difficulty using different drawing methods.
As shown in Figure 13,
hand drawing exerted the most minor mental,
physical,
and temporal demands
and was associated with the lowest
level of frustration.
The score of our method is suboptimal.
Nevertheless,
it significantly increased overall performance levels
and decreased mental,
physical,
and temporal demand relative to the previous method.
As the following step,
the field study aim to examine the usability
and expressive ability
of the completed application among
our target users - people with a motor disability.
It aimed to assess the potential of our method
to accommodate individual differences among users.
This study tested
our completed application with target
users of eye trackers
in their natural environment,
including homes or familiar places.
Participants were asked
to create illustrations according to their desires freely,
and the data collected
consisted of these illustrations.
A survey (System Usability Scale)
was conducted to quantitatively measure the final
application's usability
after the participants finished the test.
In addition,
a semi-structured interview was conducted,
and the illustrations
created by the participants were analyzed
using thematic analysis.
People with a motor disability
who used eye trackers
as their computer interaction tools
were recruited via partnerships with the People's Hospital
of Guangxi Zhuan Autonomous Region
and the Disabled Person's Federation.
The average System Usability Scale
score among the participants
was 82.75, labeled as“very good."
It indicates that our application was easy to use
and enabled target users to emulate hand drawing.
All the participants succeeded in implementing
free-eye drawing with our application
producing expressive visual works.
The illustrations created by the participants
included illustration art,
simple
line drawings, and cartoons.
Despite being affected by
different drawing skills among the participants,
the drawings were satisfactorily executed,
and no previously
identified problems with gaze interaction were discovered.
This is an example of using Eye Compass 2
to create illustrations by one of the participant
in the field study.
This paper presents gaze interaction design
to enhance free-eye drawing
and develop professional free-eye drawing software
with abundant drawing functions.
It also examines and analyzes the usability,
including the performance in creating
high-fidelity graphics,
eye movements,
and user experience
with the implementation of a completed application.
Through the improvements of this paper,
the application achieves positive usability
and sketch-level performance,
similar to hand drawing.
It also conforms to the physiological habits of
hand drawing and achieves eye comfort.
Our future work will focus on
(1) integrating interactive beautification systems
into gaze control drawing;
(2) exploring various gaze
control modalities for gaze interaction;
and (3)
developing systems based on deep learning methods
to address eye tracking errors.
Thank you!