Hi everyone and welcome to my talk on “Visualizing Event Sequence Data for User Behavior Evaluation of In-Vehicle Information Systems”
I will present our approach on how large interaction data that is collected
from production line vehicles can be visualized such that UX experts
can efficiently and effectively incorporate the data in their design processes
Why is this important at all? The systems we see
in modern cars are larger, more complex, and probably more distracting than ever
with large touchscreens becoming the main control interface.
Last year at AutoUI we presented an interview study in which we found that automotive UX experts
require data-driven support. Whereas a lot of very valuable and much
needed, research is carried out using driving simulators
or small scale naturalistic driving studies, detailed user interaction data and
riving data collected from production vehicles is either not available
or not sufficiently analyzed in the OEMs such that it can
be an integral part of the UX Design Process
In the following, we will present a first exploratory approach
that consists of different User Interaction Visualizations and builds upon
an existing telematics framework of our research partner
The system architecture consists of three major parts.
The In-vehicle Logging Mechanism is available in all new generation cars of our research partner
nd enables data collection from the vehicle bus as well as
from the HMI Interface. The collected data is then transferred
over the air to the big data platform where it's further processed and stored
The User Behavior Evaluation Module then accesses
the HMI and CAN data, merges both data sources, and computes
the visualizations I will present in the following.
We processed event sequence data as well as speed, steering, angle and gaze behavior data
In total we analyzed 27.000
Or roughly 28.000 sessions, generated by around 500 test vehciles
Our approach aims to benefit a holistic user behavior evaluation
by visualizing different levels of abstraction of user/system interaction
with the touchscreen interface. We, therefore, decided
to design these three different visualizations as each of them
answers different questions relevant to UX experts.
The Task Level View allows UX experts to inspect how
users navigate within the system, what the main interaction flows are, and how they relate to each other.
The Flow Level View provides a quantitative comparison of different flows based on a chosen metric.
And, finally, the Sequence Level View enables UX experts
to analyze certain sequences regarding the interrelation between touch
interactions, glance behavior, and driving parameters.
To better understand the visualizations, I will introduce in the following, we will use
The route guidance task as an example
We will now show on specific flow on how this task can be completed.
So, this is the first screen and when
Tapping on the navigate to button a new screen is presented.
We can now start typing and if we just choose the first
Element out of the list of suggested destinations
A new screen is presented. The last action is now to check the destination
And press the "Start Navigation" Button
Looking at this example, we can already see that there are multiple different possibilities on how the
Route guidance can be started. We decided to choose
The keyboard typing, but we can also choose the tab
To choose one of the previous destinations or favorite destinations
To visualize these different opportunities and create an easy
to grasp overview about how drivers navigate within the
system we introduce the so-called Task Level View
We remember that for this example we defined the
oute guidance task to consist of all sequences that start with
the “NavigateToButton” and end with the “StartNavigationButton”
The Task Level View is an aggregated visualization of all sequences
that belong to a specific task. The flows
are visualized in form of an adapted Sankey diagram with color-coded links
to visualize the temporal dependency between different steps
The main components are Nodes and Links. Each Node
represents an event at a certain step in a task and each link connects
two nodes representing a transition between two events
So this is the Task Level View for the Navigation Task.
We can already see that we have a lot of different flows, that in the end
belong to the same task. We will just take look at the
Flow we already walked through using the screenshots.
All flows start as defined with the "NavigateToButton". We see that we have roughly
1000 sessions. After tapping on the "NavigateToButton"
Roughly 62% of all pleople
start typing on the keyboard. We can see that when hovering
Over this transition from the "NavigateToButton" to the "OnScreenKeyboard"
This basically represents all taps that are performed on the keyboard
After typing on the keyboard, which takes all the users
on average 11 seconds, which is displayed here, when we hover this
transition users choose something out of a list
And afterwards they need on average 3.5 seconds to
check that the right destination was chosen and to start the route guidance
with the "StartNavigateToButton"
We can already see the different flows, how they are distributed and what is the preferred way of users to start the navigation.
For example, 62% of the users decide to enter the
destination via typing on the keyboard, whereas
28% decide to choose the destination out of the
previous destinations list and 7.2% percent
decide to choose the destination out of their favorites.
Having now experienced the Task Level View, we continue with the Flow Level View
Whereas the Task Level View provides an overview of the flows and how
they are distributed, a comparison between flows with regard to a certain
metric is not sufficiently visualized. We, therefore, created the
so-called Flow Level View, that shows the distribution of a certain metric of all sequences
that belong to the same flow. The different flows, that
belong to the same task are shown next to each other to facilitate
comparison between them. We decided to use violin plots as an
enhanced form of boxplots since they make it easier to assess
the distribution of datapoints, in this case the individual sequences.
If we now take a look at the example view, where we compare the different
Flows with regard to the time on task: The blue violin
The flow, that we walked through in the example
when users type on the keyboard. The orange flow represents user who choose the
Previous destination option. The green violin
represent the favorites option. We can already see that
It is way faster to either choose the favorites or previous destination option.
Which is quite obvious. But it's also 1.8 seconds faster to
choose the destination out of the favorites, compared to choosing it out of previous destinations
Whereas the Task and Flow Level View both show aggregated
statistics to infer general trends regarding user behavior.
The Sequence Level View in contrast aims to generate insights on the level
of individual interactions and individual sequences.
In addition to the interaction sequences and the glance behavior, also driving behavior
Is visualized for those specific situations.
The example shows three different situations and each
Visualization consists of two main parts: The upper part is an overlay
of touchscreen interactions (blue dots) and the driver’s glances toward the
center display (orange lines). Each dot represents one interaction
and each line indicates the duration of a glance toward the display.
The lower visualization, consisting of two graphs, represents the driving-related
data, the vehicle speed in green and the steering wheel angle in red.
Now, we will take a look at one specific example in more detail.
In this example, we see a specific sequence of the route guidance task.
We can clearly see that the head unit glance starts here at zero
and takes around 2.9 second, which pretty long.
After roughly 2.5 seconds the "NavigateToButton" was pressed.
Then, the driver looks away from the head unit
and a long glance starts and the driver starts typing on the keyboard
Checks the road again, types on the keyboard, then a long glance
takes places. Then the user drags on the list
and chooses something out of the list and finally starts the navigation.
As one can see those are pretty long glances
The speed decreases during the time and we can also see some steering wheel reversals.
Since this is a first explorative approach
we were interested in how they interact with the visualizations,
what needs to be adjust and what they think is missing
We, therefore, recruited 4 UX experts from our research partner
and after an initial walkthrough we asked them to perform 7 different tasks
and to answer a survey afterward. The general feedback
was very positive and the participants expressed that they see multiple use cases for such
visualizations in their design process. The Task Level View was considered
most helpful and served as a kind of anchor point during the different tasks.
The Flow Level View has proven to be very important for comparing individual
flows quickly at a single glance. However, two participants
struggled to interpret the violin plots.
The Sequence Level View was considered very important and participant valued
the possibility to show some example sequences arguing
that it benefits their overall understanding of how people interact with the system.
That's all and I'm now happy to answer your questions!