Rethinking Eye-blink. This work is by Youngjun Cho from University College London.
In this work, we are concerned with Mental Workload Assessment through Physiological
Computing. Mental workload that can be defined as the Cognitive demand of a task has been
extensively explored across fields including human computer interaction. Mental workload assessment
is obviously one of the important tasks in usability and accessibility evaluation.
With Self-report questionnaires such as NASA-TLX, physiological measurements and their metrics have
often been used for mental workload assessment. And here, there could be two major types of
studies to be considered. The first type is to investigate the casual effects of cognitive load
factors on physiological metrics. The other type aims to estimate mental workload parameters such
as expected task difficulty through supervised machine learning of physiological measures.
Our main consideration is spontaneous eye-blink responses to task difficulty.
This is because existing eye-blink measures appear to play a very minor role
in the first study type and much less explored in the second type as well even though they can
be easily measured using a webcam in our daily life, not requiring a specialised physiological
sensing device. Earlier studies have explored this with two key standard metrics, Blink rate
and blink duration. There have been inconsistent findings on the relationship between blink rate
and task difficulty levels. Also, the metrics are shown to be sensitive to perceptual load such as
visual stimuli, rather than task difficulty and they are prone to their interaction effects. So,
this has been the main challenge in bringing blink-measures to the mental workload
estimation task. Here, we wanted to revisit and address this problem, proposing Rethinking
Eye-blink which is a novel framework for automatic task difficulty estimation through
physiological representation of spontaneous blinking. The framework consists of three stages.
The first two are for the physiological representation to replace existing metrics
and to improve their sensitivity to task difficulty. In detail, the first stage is
to extract raw blink timeseries through face landmark detection. And in the second stage,
our blink spectrogram generator converts the timeseries to time-frequency spectrograms.
On the right side is a video to show blink spectrograms generated in real time. Finally,
the last stage is feature learning with a two dimensional Long Short Term Memory neural network
to achieve a strong non-linear mapping from blink spectrograms to difficulty-related parameters.
With the proposed framework, we conducted the two aforementioned studies. In the first study where
we used a mental arithmetic task, one of the most standard ones in exploring task difficulty effects
in many fields, we revisited the sensitivity of the standard blink metrics to understand the
extent to which a blink representation improves its sensitivity. Here we used blink entropy
to quantify our blink spectrogram. And the result has confirmed that the representation can improve
blink measures' sensitivity and highlighted the importance of exploring time-frequency
representation of eye-blink when capturing its response to task difficulty.
From the task difficulty estimation study using leave-one-subject-out
cross-validation to test the generalisability, our method produced state-of-the-art accuracies,
outperforming the other baseline approaches that used the hand-engineered blink metrics. This
comparison result confirms that the sensitivity gain and improved performance are indeed from the
more powerful representation, rather than the more powerful machine learning classifiers.
Lastly, we highlight some benefit from machine learning.
Pre-trained models in lab settings are scalable in real-life situations where datasets grow.
Having said that, here we consider some of the real-world applications,
like usability evaluation for websites and accessibility monitoring in VR applications.
That's the end of my presentation, please check out the paper for more details.
And the source code is also available on Github. Thanks for listening.