Label likelihood maximisation
Adapting iris segmentation models using domain adaptation
Data is almost always the limiting factor for machine learning researchers and engineers.
Data is expensive to collect and it is hard to create well-balanced datasets.
A lot of research has gone into changing this either by generating data or by providing
extremely large-scale datasets.
The problem is similar to the chicken and egg dilemma.
Data is essential for model development and great models are essential for improving data
collection quality and volume.
We approach the problem by using priors to improve model predictions.
This makes model self-improvement possible.
Our contribution is a general principle for applying priors as well as an implementation
for an iris-segmentation problem.
Datasets are samples of data that are drawn from an underlying distribution.
This distribution determines the characteristics of the data and is therefore called its domain.
For eye-tracking, each setup may define its own domain.
For example, a change in lighting conditions may change the general appearance of samples.
In a perfect world, one would find a domain for eye-tracking that covers all such variations.
In reality however, complete coverage is not realistic due to the complexity of the domain.
Therefore, the domains of the model training data and model test data are different.
This is known as domain shift.
Domain shift can be alleviated by applying techniques that modify the model in some way
as to adapt it to the new domain - this is called domain adaptation.
We perform domain adaptation by generating pseudo-labels for unlabelled data.
The base task is semantic segmentation of the iris which is achieved using an encoder/decoder
neural network architecture.
The network used is an FCN variant similar in design to DeconvNet.
The network consists of a number of encoding blocks, an encoded representation, and a corresponding
set of decoding blocks.
Each block consists of convolutional layers and either pooling or unpooling operations.
Pooling indices from the encoder allows spatial information to pass easily to the decoder.
Label likelihood maximisation (shortened LLM) is a method for generating pseudo-labels
that are used to train a model on data from a new domain.
The method uses the current model and a label prior to create pseudo-labels and is therefore
entirely unsupervised.
Regular gradient-descent based training proceeds by generating predictions and calculating
the loss to update the model.
With LLM, pseudo-label generation replaces the ground-truth label when calculating the
loss.
Pseudo-label creation is based on a few assumptions.
Specifically, it assumes that the model is already at least somewhat competent in making
predictions even under domain shift.
Additionally, it assumes that the prior distribution of labels can be modelled realistically.
For the task of iris segmentation, the prior is a uniform distribution of all valid
ellipses drawn in the 224x224 output space.
The true prior is not known, and the one chosen here is therefore just a model choice (albeit
an informed one).
This prior defines a manifold in the output space.
The starting point is the model prediction.
Predictions are assumed to be distributed around the true value according to some distribution.
We here use the normal distribution.
LLM optimises the likelihood of getting a specific prediction by finding the closes
manifold point and assuming that to be the true label source.
Because the manifold is not linear, predictions are ranked by the distance between prediction
and manifold.
This represents the model's confidence in the pseudo-label.
Since LLM is used for adaptation of base models, the following experiments focus on
their relative performance.
The base model is trained using synthetic data from the UnityEyes generator.
For testing and domain adaptation, a collection of four real-world datasets is used, totalling
almost 300.000 images.
The base model is adapted to each dataset separately.
The datasets only contain pupil centre annotations which are approximated using the
predicted iris centres.
The centre is extracted from the segmentation by performing the same LLM procedure on a
prediction and then extracting the parameters of the resulting ellipse.
This is important because it ensures that no advantage is given to the immediate improvement
caused by segmentation post processing.
We use the detection rate as a function of maximum distance error to evaluate the
results.
Base is the same model on all graphs while Adapted is the individually LLM adapted model
for each dataset.
On all but one dataset, a clear relative improvement is present.
The last graph shows slightly worse detection rate at low distances and slightly better
detection rate at high distances.
Some example images show that the models do indeed improve their ability to create
realistic iris segmentations for the provided samples.
However, it is also clear that the size is not being inferred correctly.
Considering the size distribution for the ellipse prior may help alleviate this problem
in future studies.
The results are significant because LLM provides a considerable relative improvement
at no cost - it is free to use because it doesn't require annotated data.
And because all models used the same ellipse-extraction method for centre estimation, the relative
improvement must necessarily be caused by improvements to the model itself and not by
post-processing gains.
The work presented here suggests many possible paths for future studies.
The core takeaway is that model knowledge is important and does make a difference when
creating neural-network based eye-tracking systems.