Hello everyone! My Name is Thomas Kundinger. Now I will give
you some insights about the work I did together with Ramyashree Bhat and Andreas Riener with
the title “Performance and Acceptance Evaluation of Driver Drowsiness Detection System based
on Smart Wearables” Drowsy Driving is still a major reason for
many fatal traffic accidents. A high number of crashes, e.g., in the US or Europe is caused
by drowsiness. So, we can see, that there is a need for driver state monitoring, that
means, a need for systems that can reliably detect driver drowsiness and warn the driver
in time to prevent accidents. Apart from the evidently need, there is also
a duty for driver state monitoring systems for automobile manufactures through external
requirements, e.g., the EU General Safety Regulations. Regarding this law, from July
2022, all new vehicle types must be equipped with a DDAW, a Driver Drowsiness and Attention
Warning System. And from July 2024, all vehicles to be registered need to be equipped with
such a system. What is expected and said to have an even
higher impact on road safety and reduce human-made errors, e.g., through drowsiness, is automated
driving. Based on the SAE, the automation of the driving task is categorized into different
levels, from manual driving/no driving automation, SAE level 0, to the full driving automation,
SAE level 5. Across these levels, the driver's role changes
from the sole operator in SAE level 0 to the fallback level in SAE levels 1-3 and finally
to the passenger in SAE levels 4-5 of an entirely automated system. Therefore, and in terms
of driver drowsiness, the lower levels of automation, namely SAE level 1, 2 and 3 require
special attention. Due to the decreased active involvement in driving, the risk of getting
drowsy faster is increased. Therefore, in the transition phase from manual driving to
the rollout of full automation, systems that reliably monitor and recognize the driver's
current state, particularly the risk factor drowsiness, will be necessary to guarantee
the driver's ability take over full control. So far, different measures have been proposed
to detect drowsiness in an automotive environment that all have different advantages and limitations.
These measures can be categorized mainly into four groups: subjective, behavioral, vehicle-based
and physiological measures. Subjective measures apply questionnaires.
They take the personal feeling into account but cannot be used in real-time or a real
driving context. Behavioral measures evaluate eye movements,
facial expressions, and head position by using a driver monitoring camera. These measures
are non-intrusive but interpersonal accuracy or privacy issues can limit the detection
performance. Current commercial systems mainly focus on
vehicle-based measures, that means, the analysis of parameters related to the driving behavior,
such as lane position or steering angle. These measures are non-intrusive. However, with
increased driving automation, these parameters will be more difficult to evaluate since the
automated system controls the car. Therefore, alternatives are necessary.
A promising alternative seem to be methods that evaluate physiological data to identify
driver drowsiness since this data can be used to differentiate between wakefulness and sleep
and warn the driver in an early stage. However, typical EEG or ECG measurements including
electrodes on the head or upper body, are not accepted inside a vehicle. Therefore,
new non- or less intrusive strategies for recording physiological signals inside a vehicle
are required what was investigated in this work.
To counteract limitations of existing measures our approach was to use physiological data
of smart wearables, such as smart watches and fitness trackers, for driver drowsiness
detection and investigate their suitability in an automotive environment.
Apart from their actual use in daily health and fitness monitoring, by using these devices
inside the vehicle, their scope of use could be further enhanced. This kind of driver-vehicle
interaction offers the option of being easily integrated into a vehicle. Existing vehicles
can quickly be upgraded and retrofitted with this technology without having to install
additional sensors. Since these devices can be worn like a watch, their usage inside a
vehicle would enable long-term recording of physiological signals in a non-intrusive and
driver-familiar way. We investigated the potential and feasibility
of the recorded physiological data, in our case heart rate, of a wrist-worn smart wearable
as single data input for the detection of driver drowsiness in combination with supervised
machine learning. To broaden the picture and since drivers are kind of “forced” to
use a wearable device while driving, in this context, we also examined how the acceptance
of drowsiness detection systems based on smart wearables is.
This resulted in the following hypotheses. H1: A high detection accuracy can be obtained
by using exclusively physiological data (heart rate) from a wrist-worn wearable device for
driver drowsiness detection. H2.1: There is a high acceptance after first-time
use of the proposed drowsiness detection system. Since older people are a potential target
group of automated driving, we focused in the evalution also on the age of the drivers
and compared older with younger ones. This resulted in the last hypothesis:
H2.2: There is a significant difference between younger and older people regarding the acceptance
of the proposed drowsiness detection system. To investigate the mentioned issued, a prototype
for driver drowsiness detection with a smart wearable device was developed. From a Polar
device heart rate data is streamed with Bluetooth Low Energy to an Android app on a tablet.
On the tablet, the data is pre-processed, features are extracted and fed into a trained
random forest binary machine learning classifier that calculates the current drowsiness state
of the driver. Regarding the user interface, a total of six
different screens were implemented. Screen 1 shows a “welcome screen” after starting
the app. Screen 2 contains a user manual with pictures of the wearable, how to put it on
the wrist, switch it on and connect it with the tablet. Once the smart wearable is connected,
the current state of drowsiness is presented, either “non-drowsy” (screen 3) or “drowsy”
(screen 4). When the drive is finished, screen 5 appears, showing the current battery status
and indicating that the wearable should be recharged until the next drive. Within the
scope of the user study carried out in this work, the drowsiness state was not presented
to the participants while driving but only stored in the background. In return, the subjects
had to assess their drowsiness via self-ratings every five minutes. For this purpose, the
Karolinska Sleepiness Scale, ranging from level 1 (extremely alert) to level 9 (very
sleepy, sleep fighting), was displayed to the participants, as shown in screen 6. The
reason behind this was to compare the output of the machine learning model with the subjects’
self-ratings to evaluate the performance of the machine learning model.
To evaluate the mentioned issues, a user study in the hexapod driving simulator at Technische
Hochschule Ingolstadt was conducted, as shown in the left picture. In the center picture
you can see the tablet pc with the android application in the center console. The right
picture shows the wearable device on the participants wrist. In the beginning, the participants
got an introduction, filled a questionnaire, and had some practice driving in the simulator
to get a feeling for it. Afterwards the main part of the experiment started, a 60-minute
SAE level 2 drive, where the participants had to enter, every five minutes a drowsiness
self-rating. After the study, participants answered the item of the technology acceptance
model, TAM, filled the post-questionnaire, and had a short interview with the experimenter.
In total, 30 participants from two different age groups, three per day, participated in
the study. Regarding detection performance and hypothesis
1, we achieved on average across all participants an accuracy of 82.72% with completely new
and unseen data. It can be seen that a high inter-driver variance exists ranging from
50% up to 100% and the model did not perform equally well for all participants. Focusing
on the two age groups, the classification accuracy is around 15% higher among the older
subjects. Therefore, hypothesis 1 can only be partly accepted. All in all, these are
promising results concerning the early development stage of the model and small training data
set. Regarding technology acceptance, in this figure,
a summary of the results of the different TAM subscales for younger and older participants
is presented. High ratings for technology acceptance with significant differences between
the age groups with higher ratings for the older participants were found. Therefore,
both hypothesis 2.1 and 2.2 can be accepted. The results from the interviews after the
study showed, that the design of the user interface needs to be both appealing and above
all understandable for different age groups. Especially for older participants, the user
manual for handling the wearable device was of great importance.
Of course, our work also has some limitations. Regarding Limitations, Specific precautions
were taken for a faster increase of drowsiness. Since we conducted a simulator study, there
was no real risk for participants and therefore also a potential faster increase of drowsiness.
Regarding prototype evaluation: This was a single session experiment and no evaluation
of long-term experience/usage. Further, the focus was on two specific age groups
Future work includes a follow-up study where the drowsiness level is presented to the driver.
We also plan to integrate a warning into the prototype. Data fusion with other sensors,
e.g., a driver monitoring camera, will be considered. Further, the interface design
will get an update based on the feedback of the participants and more training data will
be collected to increase the detection performance of the machine learning model.
Thank you very much!