Welcome to the the virtual presentation of “Self-interruptions of Non-Driving Related
Tasks in Automated Vehicles: Mobile vs. Head-Up Display”. My name is Michael Gerber and
I have the honor to present this research in behalf of Dr. Ronald Schroeter, Dr. Xiaomeng
Li and Dr. Mohammed Elhenawy, all members of the Centre for Accident Research and Road
Safety – Queensland and the Queensland University of Technology.
Currently we are driving vehicles still by ourselves. Research indicates that drivers
are engage in secondary tasks with increasing automation. This can lead to dangerous behavior
and therefore are some non-driving related tasks are forbidden, such as the use of a
mobile phone in most of the countries. Let’s start for the beginning. The Society
of Automotive Engineers, short SAE has developed a standard for common understanding to describe
the transition from manual driving to full automation in levels while increasing the
automation in each step. In this description the first 3 levels from
0 to 2 are considered as manual driving. These can include automated driving features such
as advanced distance cruise control and line keeping.
During this 3 stages the driver performs the driving task as primary task.
A secondary task is therefore considered as distraction.
This will change in the future with the development of automated driving systems, they will free
the driver from the driving task and enable so called non-driving related tasks. But we
just can fully engage in NDRTs until we have reached the stage of full automation we need
to go through the progress of conditional automation and high automation.
In these a driver needs to remain fallback ready, to overtake vehicle control if the
vehicle reaches a limitation of its capabilities. The implication to remain fallback-ready can
be a change to a lower level, such as partial automation, which means the fallback ready
driver changes its roll to a driver. This fallback-readiness reduces therefore
the freedom of the NDRT and can be even safety critical. Our focus is therefore in Level
3, Conditional Automation and Level 4, High Automation. The results we found are also
important for Level 5 to address further issues in Full Automation.
This driver state transition is described in a framework from Janssen et al. It describes
the transition from the non-driving task to the driving task, which can be safety critical
and afterwards the change back to the non-driving task. In this research we are focus on the
driver state transition from the NDRT to the driving task. Janssen et al. are using steps
to describe schematically how humans changing their state.
While in 0 the fallback driver is engaged in an NDRT, self-interruptions occur during
that time, or an external alert triggers the fallback.
Self-interruptions in this context is a voluntary interruption of the NDRT. In contrast an external
alert is involuntary. This can cause a disengagement from the NDRT,
followed by an orientation phase. After the non-driving task is completely suspended,
and the physical transfer of the control is finished, the driver is able to contribute
full to the driving task. However, these stages 2-5 are interleaved and can occur more often
in the driver state transition. In this research we focus on the self-interruption of the first
step. Because following the hypothesis from Schroeter and Steinberger, that an increased
Situation Awareness leads to a reduced take over time. In Janssens framework the reduced
transition time of the driver state transition is caused from a higher Situation Awareness
reduces the orient phase which reduces therefore the interleaving time. There are important
other factors included in this calculation as well which are described in a driver-state
transition model from Marberger et al. However, in this research we are focusing
on the self-interruption during the non-driving task. This means for us, we are reducing the
framework to an interleaving process of the NDRT and monitoring, which might end up in
a voluntary disengagement. Our Assumptions is, that the conditions of the NDRT are changing
the behavior of Self-interruptions. This Assumption is grounded in Wickens SEEV Model of Visual
attention allocation. SEEV stands for salience, expectancy, effort and value. The focus for
this study is on salience and effort. This brings us then to the research Question
for our publication: How do different display devices a) affect
long NDRT fixations in the NDRT and, b) self-interruption behavior during automated driving?
To explore different NDRT conditions we choose to change the condition of display devices
and have decided to compare mobile phones, who are well integrated in our everyday live
with a novel Head-up Display. We have used a within-subjected research design
and counterbalanced, with the first condition as head-up display and the second condition,
mobile phone. As non-driving related task, participants have been allowed to choose an
entertaining video content from a public tv-stations online media library. The study took place
in the High Fidelity Simulator of CARRS-Q and QUT. Compared to previous studies we used
to simulate automated driving real world recorded videos and played them back on the front screens,
side and back mirrors and showed the GPS-route on a mobile phone in the center of the vehicle.
The first condition of the Head-up Display was shown as a transparent, fixed overlay
on the center screen to mimic a windshield display. In this analysis we used head and
eye-tracking to label NDRT engagement or interruptions. In the second condition participants have
not given restrictions or hints how to place the mobile phone to achieve a more realistic
behavior. This condition required therefore manual annotation. Each drive took 15 minutes
and started in an urban area, continued on a Highway and ended in a suburban area. In
this data analysis we have labeled 5 minutes of “uneventful” automated driving at the
end of the highway drive, before the vehicle changed to the exit lane. The label are “Number
of long Non-driving related Task fixations”. Every time we talk about long fixation, we
talk about fixations over 2 seconds. This timeframe is typically used in distraction
research. “Duration of long NDRT fixations”. “Number of Self-interruptions” and duration
of self-interruptions. We invited 42 participants, 23 male and 23
females. They aged between 20 - 62, with an average age of 32 and a standard deviation
of 8.4. All participants held a valid Australian driver’s license. Let’s have now a look
at the results. All results in this presentation are shortened,
we encourage the audience to consult the paper for more insights.
At first, we are looking at the yellow graphs. The filled line shows the mobile phone condition,
while the dotted indicates the Head-Up condition. When we compare these, we see then, that the
mobile condition has a higher number in engagement on NDRT compared to the Head Up Display.
The mean number of long NDRT fixation where in the mobile condition 16.3 with a standard
deviation of 8.9 while in the HUD condition the mean value was at 3 with a standard deviation
of 4.49. Now we have a look at the blue graph. Again,
the filled line shows the mobile phone condition, while the dotted indicates the HUD. The HUD
condition indicates a significantly reduced expected engagement duration on NDRTs in seconds
compared to the mobile condition. The expected engagement duration of an NDRT
was in mean 18.5 with a standard deviation of 18.6 compared to the HUD 2.76 with an SD
of 0.48. If we look now at the Self-interruptions.
The number of self-interruptions are 10.9 with a standard deviation of 7.9. In the HUD
condition, the mean was 23.04 with a standard deviation of 7.04.
The duration of a self-interruption took in the mobile condition 6.27 with a standard
deviation of 5.36 while in the HUD condition a self-interruption took in mean 3.9 seconds
with a standard deviation of 1.69. To summarize the results:
We compare mobile vs Head-up display Long NDRT fixations are in the mobile condition
higher and less in the HUD condition. A long NDRT fixation was longer in the mobile
phone condition and shorter in the HUD condition. Self-interruption occurred less in the mobile
phone condition then in the HUD condition And lastly the interruption took longer in
the mobile phone condition as in the HUD condition. So what do we learn about this?
Self-Interruptions in automated driving from the NDRT might has safety critical benefits.
Let’s remember the visual allocation model SEEV.
We somewhat controlled expectancy and value of the TV show and driving scene, while driving
the same route and let the participants choose TV-shows by themselves.
When we have now a look at the results, we could interpret the changes with salience
and effort. Overall, the effort involved in retrieving
environmental information is lower in the HUD condition, while the salience of the environmental
information is higher and versa visa. Switching between NDRT and monitoring takes
effort. We interpret less effort in shorter interruption times which we can see in the
self-interruption duration. This effort includes also physical changes,
such movements as raising the head. That participants are having less interruptions
in the mobile condition indicates, that they are more engaged in their NDRT and the effort
might be to high to interrupt it more often. This is then reflected in the fixation duration.
Participants seem somehow to forget to be in an automated vehicle, even if we told them
before that they have fallback ready functions. This is then reflected in the number of long
NDRT fixations as well. Through the study shows that younger drivers
have longer NDRT fixations durations in contrast to older drivers. Older driver seem to interleave
tasks more often. Generally driving experience increases with drivers age, which could indicate
that experienced drivers tend to be more vigilant at the road and traffic environment.
This might have implications for future attention management systems to keep drivers fallback
ready. Summarized, the results indicate, as hypothesized
by Schroeter et al., that the visual integration of the NDRT in the driving scene aids self-interruptions
and task-interleaving is attributed to the to the driving task.
As every study, this research has limitations as well. These are amongst others:
As mentioned, we have used two different approaches to label the conditions, to provide participants
the freedom, to have their mobile phone at a position of their own choice. The HUD or
Windshield display was a simple overlay, instead of a 3-dimensional windshield display. And
as Shadans Borojeni et al. research indicates, that motion influences the results of studies
like this, we haven’t used motion during the video drives. We investigated how people
may interact with automated vehicles in Level 3 and Level 4 while engaging in NDRTs. Especially
how driver's discretionary self-interruption behavior during that time.
We present significant differences in frequency and duration of such interruptions, that suggesting
a higher likelihood of a higher situation awareness when NDRTs consumed over a HUD rather
than over mobile phones. This was Self-interruptions of non-driving
related tasks in Automated Vehicles: Mobile vs. Head-up displays. Thank you for your attention.
If you have questions, please find here our mail, we are happy to get in contact with
you.