Good morning, good afternoon, and good evening everyone.
My name is Ryan. I'm a PhD student at UNSW Sydney and today,
I'm going to present EyeXplain Autism: an interactive system for eye tracking data analysis
and deep neural network interpretation for autism spectrum disorder diagnosis.
Just a brief outline of today's talk.
I'm going to introduce the research aims and objectives, followed by the methodology.
I'll present EyeXplain Autism interface and discuss the limitations,
our ongoing work and finally conclusion.
Autism Spectrum Disorders or ASD are neurological disorders characterised
by deficits in cognitive skills social and communicative behaviours.
They are currently diagnosed through manual coding of behaviours mainly by clinicians.
In Australia, 1 in 70 people or around 354 000 are on the spectrum.
In our current review of related literature, we found that different
biological and behavioural markers can be quantified using computer vision.
These markers include magnetic resonance imaging, facial expression and emotion, eye gaze data,
motor control movement, stereotype behaviours and finally a combination of any of these.
In this talk, I'm just going to focus on how eye tracking data can be used for ASD diagnosis.
Recently, deep neural networks have seen
tremendous progress in their ability to diagnose autism.
However, they are often seen as black boxes by physicians unfamiliar with it.
That's why in this research, we're going to present EyeXplain Autism
and discuss a sample scenario where clinicians can analyse eye tracking data,
perform automated diagnosis and interpret DNN predictions.
In this talk, I'm not going to focus on clinical protocol and model training.
Please refer to the paper for more information.
In general, there are 17 typically developing and 17 ASD individuals and we computed for
the difference of fixations per frame to come up with a visualisation like this.
We trained a rapid end-to-end saliency prediction model
to then predict those difference of fixation maps.
Once the model has been trained, we upsample the data from the convLSTM output and then
extracted features from the fixation locations recorded during the experiment.
We then appended k number of fixations which empirically we found that k equals 13 would
yield the optimal performance in terms of specificity, sensitivity, and accuracy when
we use a support vector machine to identify autism and typically developing.
Now for the EyeXplain Autism Interface, here's the website.
On the top of the screen, you'll see
the control panel which would allow you to load an eye tracking data, load stimulus,
switch between the visualization and the statistics tab,
sort fixations and do an automated prediction.
I'll start off by loading a stimulus and loading an eye tracking data file from an autistic individual.
Here you can see in the middle is the visualisation tab which would allow
you to view the stimulus and as well as view the recorded fixation shown here by a green rectangle.
You can also visualise the ASD heat map which is the aggregate of all the ASD
samples during training as well as the TD heat map which is a typically developing
and the difference of fixation heat maps
On the bottom of the screen, you'll see the interactive timeline which
shows the chronological sequence of all the recorded fixations.
When you hover into a fixation, you'll see a randomly sampled ASD and TD sample and when
you click to a fixation, the visualisation tab will jump to that recorded fixation.
You can perform automated diagnosis by clicking the predict button.
And this button is connected to a CherryPy server
which contains the model that I just discussed before.
To further analyse and interpret the prediction,
you can click the sort fixation button, which sorts all the fixations by feature importance.
So when you analyse the difference of fixation maps of the top five fixations,
you'll see that the recorded fixations or the green rectangles lie on the ASD heat map.
You can further analyse the eye tracking data using the statistics tab
and comparing them to a pre-computed values of the samples seen during training.
Now, the limitations of this is that we used a relatively small data set of about
34 individuals and we only evaluated only one saliency prediction model.
Our ongoing work includes incorporating severity and treatment response prediction,
providing flexibility to use different models for prediction
and finally an evaluation with clinicians.
I want to acknowledge our collaborators from UNSW psychiatry,
the children their families for their participation and that's it.
Thank you!