I'm Mel Arthurs
one of the authors of this paper
on the challenges of developing a mobile
psychomotor vigilance task,
also known as the PVT.
PVT is a test commonly used to assess
participant’s cognitive fatigue.
Reaction times are analysed to predict participant alertness
and therefore cognitive fatigue.
Three phases contribute to the participant reaction times
detecting the stimulus,
recognising it as the stimulus,
and reacting to the stimulus
using the touch down gesture on the device touchscreen.
The participant completes multiple PVT trials
with an interstimulus interval.
In this case, we used a random duration
between 2000-3000ms.
To offset the less accurate timing produced
trial count can be increased
but with the consequence of increasing participant burden.
The ideal mobile PVT solution
would reduce participant burden compared to traditional lab based methods.
It would aim to be as precise as the lab based methods.
The mobile solution would have the advantage of being more ecologically valid
as participants could complete the test often and regularly
in a place of convenience.
Three frameworks were used for the iOS implementations -
MetalKit, UIKit, and WebKit.
This allowed us to compare the RT differences across common iOS frameworks.
We created and validated an Arduino based test instrument
to test these implementations
where the Arduino simulates a user with a 250ms reaction time.
This would be the 'ideal' user
as 250ms is the average human reaction time.
We validated the instrument using a microphone
to detect the touch of the relay on the screen.
In the future we hope to extend these results
to include android implementations as well.
We defined a number of key timing measurements
to better identify the bottlenecks in the measurement process.
The PVT reaction time is the reaction time measured by the PVT implementation.
The implementation latency is the sum of CPU to Display Latency,
the Input Latency,
and the Timestamp Recording Latency.
CPU to Display Latency is the time taken to execute a graphical update on the screen.
And finally,
input latency is the time taken to register the user’s touch on the screen
and call the touchesBegan method.
Moving onto the results,
We see an average latency of ~74ms for the MetalKit implementation
over 100 trials.
We were able to measure the CPU to display latency
with an average of ~19ms
and a standard deviation of ~12ms.
Compared to the input latency
the cpu to display latency is a significant cause of variability.
Implementation latency increases for the UIKit implementation
with an average of ~83ms over 100 trials.
Again, input latency is negligible here.
We were unable to measure the cpu to display latency
for both UIKit and WebKit implementations.
Finally, for the Webkit implementation,
the average implementation latency is the most significant
at ~121ms over 100 trials.
Looking at the 100 trials over a line chart
the precision issues are clear,
with large variations in the PVT reaction time for every implementation.
From these results,
we can conclude that mobile PVT results from iOS are overestimated by up to 159ms.
Results are affected by the software implementation.
and that the results are affected by the device configuration
In this paper,
we also provide an accessible method of determining these effects for other implementations.
Thank you for listening.