DeepTake, Prediction of Driver Takeover Behavior Using Multimodal Data
I'm Erfan Pakdamanian and I'm presenting this paper on behalf of my collaborators.
DeepTake overview
In level 3 of automated drivings, drivers are not strictly forbidden to engage in
non-driving-related tasks (NDRTs)
Drivers are still expected to intervene and take over vehicle control
when takeover request is issued by the system.
However, if the driver is engaging in non-driving activities,
there will be a sudden increase in workload and stress.
To cope with the safety challenge of of takeover, we propose "DeepTake" to reliably
predict takeover Intention, Time, and Quality together.
Since Driving is full of uncertainties and because
every possibility should be programmed into the vehicle.
Accidents and unforeseen results will happen.
Although the future will be autonomous and our quest to get there needs a transition
period to get a better integrate human and machine expertise in safety-critical situations.
Since the high level goal for automated vehicles is to be “reliable, safe and trustworthy”,
We should move towards designing “humanistic” or “human-centered” systems, to create more effective
and safer human-automation collaboration.
As drivers are allowed to engage in non-driving activities marinating them vigilant
to properly react in critical situations is crucial.
So, the question becomes: "How long it takes drivers to reclaim manual control?"
and "what factors determine a safe takeover?"
Previous studies have addressed this issue into “TWO” major areas:
One: Evaluate take-over performance with respect to:
"Takeover Time", "Takeover quality"
Some investigated predictive models for estimation of either "takeover time" or "takeover quality".
However, they obtained relatively low accuracy for on-road applications.
So, we still need an enhanced system to not only estimate all the aspects of takeover,
but also be more robust for real-world applications.
Secondly, some studies investigated different modalities.
In most of these studies, takeover request were presented unfailingly.
Therefore, such takeover request bear a little resemblance to real automated driving.
"DeepTake" on the other hand is the first framework which integrates deep neural network
model to predict driver "takeover intention", "takeover time", and "takeover quality".
We envision that DeepTake can be integrated into future automated vehicles.
Such that, the automated system can make optimal decisions based on the predicted driver takeover behavior.
For example, if the predicted takeover time is High and takeover quality is low,
the automated system can warn the driver to engage less
with non-driving tasks.
In addition, we utilized three types of alarms: "True Alarm", "False Alarm", and "No Alarm".
Because, in a precarious world, automated vehicles should be expected to fail to always provide
correct takeover requests.
So, the scenarios where constructed conservatively to include flawed takeover requests.
For the research presented in this paper, we conducted experiment with 20 subjects.
To capture their readiness, cognitive, and physiological states, before issuing a takeover request,
we used biometric signals.
Each participant drove through three trials followed by a 5-minute break between each.
Within each trial, we also designed various events.
Participants were also asked to engage with four none-driving activities.
In each trial, 15 takeover requests occur with: six true alarms, six false alarms,
and three no alarms.
45 takeover requests, in total.
All the captured data were then chunked to ten seconds before the takeover request in order
to predict what will happen after the takeover request initiation.
This chunk of data was preprocessed, labeled, and meaningful features extracted from.
Finally, this chunk of data were feed into the neural network with a three hidden layers.
To extensively evaluate the performance of DeepTake, we use multiple metrics.
And compare the results with six off-the-shelf models.
First, we compare the algorithms in correctly classifying takeover behavior.
The result shows that DeepTake achieved significantly higher accuracy as opposed to the other models.
Second, ROC curve was applied in order to better evaluate each of the classifier.
"DeepTake" also shows outperformance on correctly classifying all aspects of takeover behavior.
Finally, in order to draw better insight in performance of DeepTake, we use confusion matrix
to measure the rate of miss-classification for each class of takeover behavior.
Overall, marginal miss-classification were found in takeover intention and time.
But, more incorrect classification in takeover quality.
To sum up, with the presented novel framework, DeepTake, we extent previous literature by providing
new insights on predicting takeover intention, time, and quality together.
More specifically, we provided a reliable framework which can be applied
in real-world applications.
We also tested flawed takeover requests on takeover performance.
Thank you for tuning in!