Hello all, I would like to present our paper on At-home pupillometry using smartphone facial identification cameras.
This project is a collaboration between UCSD Design lab, the UCSD Center for mental health technology and the Digital health lab at UCSD.
Our work focuses on developing a novel pupillometry method to increase accessibility. Pupillometry is a measurement of the pupil that is commonly done as part of psychiatric tests.
The pupil is known to respond to light stimuli, strong emotions, and cognition. Recent psychiatry research also demonstrates that pupillometry is also correlated with several neurodegenerative diseases,
and there is a strong possibility that pupil measurements could lend themselves well as digital biomarkers for future disease screenings.
However, pupillometry is normally performed in-lab with clinical pupillometers that can cost around $10,000.
This creates access issues for many patients, especially older adults. Only the patients in the immediate vicinity of or the means to travel to a clinic have access to this technology.
A major part of our goal was to work towards enabling more accessibility to pupillometry.
We hope that this can aid the developing clinical field utilizing pupillometry as a biomarker for neurological diseases and eventually enable at home screening for a number of neurological diseases.
Previous attempts at smartphone-based pupillometry have been made that use the RGB camera and flash. Some works have a number of limitations including the required attachments and poor accuracy validations.
All works we could find require illumination of the phone flashlight, which limits the types of pupil response tests that can be performed.
This is shown by the on the graph on the right where the black smartphone reading stops for large sections of the test.
Lastly, because prior works utilize the RGB cameras on smartphones, they generally perform poorly on eyes with darker iris colors.
Our work seeks to address the challenges of smartphone-based pupillometry in a step towards smartphone-based pupillometers that are viable for research applications.
We repurpose the facial recognition system, which includes a NIR infrared camera and light source.
Just like clinical pupillometers, using the near-infrared spectrum allows measurements without visible light that would clearly discern between the iris and pupil.
However, repurposing the facial recognition system presents a challenge because of the low pixel density. We use an image processing pipeline and a convolutional neural network model to detect the pupil with sub-pixel accuracy.
Using the front facing RGB camera along with the NIR camera for facial recognition, we can take images from 2 different viewpoints to estimate the distance from the smartphone camera to the pupil.
To calculate the distance between the camera lens and the pupil, we compare the center of the pupil in an RGB image taken before the video to the pupil center location in the initial near infrared video frame.
It is important to recognize here that we are only using the pupil center in the RGB image. The RGB image is not used for pupil diameter estimation.
We collect data through a simple user interface with basic instructions provided by the smartphone. The collected data is processed off-device to result in both absolute and relative measurements of pupil diameter.
To validate the accuracy of our set up, we conducted a comparison to a clinical pupillometer, the neuroptics plr 3000.
In our experimental set up, the clinical pupillometer records from one eye while the smartphone records from the other simultaneously.
For the test, the smartphone administers a pupillary light response test and a digit span recall test. The pupillary light reflex test is meant to demonstrate measurements across a large range of pupil diameters
and mimic the types of testing done on previous smartphone-based systems.
However, this test can be a misleading representation of accuracy because of the drastic change in pupil diameter lessens the proportion of error between the systems.
Alternatively, the digit span recall test generally elicits small changes in pupil diameter so differences between the systems are much more evident.
In validating our system, we compare both absolute and relative measurements. The relative measurement comparison reveals the ability of our system to track
changes in pupil diameter, while the absolute pupil measurement tests the ability of our system to provide millimeter values of pupil diameter by measuring the distance
between the camera lens and the pupil using the binocular disparity.
Note that all of our validation tests were performed with no smartphone attachments or aids.
We were able to get good results from the light reflex test with a mean absolute error of 0.39mm and a median error of 0.27mm.
The mean and median relative mean absolute error was 5.02% and 3.52 % respectively. This demonstrated quality performance of the device.
In the Digit Span Recall Test performance, we expected higher absolute error, but we recorded higher error than we were expecting for both absolute and relative measurements.
This demonstrates the importance of these types of higher quality tests for pupil measurement systems. From the data, we suspect that some of the error is due to the increased length of the digit span recall test.
Since the digit span recall test is much longer the participant must hold the phone steady for a much longer time.
For the smartphone deployment, password protected google pixel 4 smartphones with our pupillometry application installed were delivered directly to the homes of study participants.
The participants received a 15 to 30 minute zoom call in which they learned how to use the device, self-administered a pupil response test, and answered questions about the experience.
In the usability study, we found that all participants positioned the phone at the eye and initiated the tests successfully.
However, we notice a few usability issues during our study, with the most common being difficulty in unlocking or turning on the smartphone, usually for first time smartphone users.
Of the 15 total participants, 8 patients had data at least 80 percent usable. 2 participants had data at least 50 percent usable.
Lastly, 5 participants had poor data quality that was mostly unusable. Most of the unusable data came from low hanging eyelids,
a problem we did not experience with the younger test subjects from our validation experiment.
Some possible improvements for our system include altering some instruction and changing the imaging angle of the device to avoid obstruction from the eyelid.
Additionally, we plan to implement iris detection rather than just pupil detection so that the iris can be used as a reference for the absolute measurement.
We suspect that this addition will greatly improve the performance, especially the absolute performance of our system during longer pupil response tests.