Hello, my name is Chris Zimmerer
and in this talk i am presenting our paper with the title:
Reducing the Cognitive Load of Playing a Digital Tabletop Game with a Multimodal Interface.
But first of all, what is a digital tabletop game?
A digital tabletop game combines aspects from video and boardgames
and to do this, they commonly utilize a large interactive surface in a table configuration.
Playing a digital tabletop game causes Cognitive Load.
In fact, performing any kind of task does.
Cognitive Load is a concept that relates the resource requirements of a particular task to our available mental resources.
On the one hand Cognitive Load is influenced by the task’s difficulty.
So for example, the woman on the left is thinking about a strategy to win the game.
On the other hand, it is also influenced by the interaction.
For example, the man on the right is trying to find a certain menu item.
Experiencing high levels of Cognitive Load is bad.
It induces stress, causes human error, and limits our ability to perform a task successfully.
Thus, the goal is to keep Cognitive Load as low as possible.
Since task difficulty can not be changed, we have to optimize the interface
and indeed previous research has shown that that decreasing Cognitive Load results in a better User Experience.
A promising interaction paradigm in this context are multimodal interfaces,
especially interfaces that use a combination of speech and touch input as depicted on the image on the left.
This combination does not only have the unique potential to keep Cognitive Load down,
but it also provides a high expressive power.
Touch input excells at providing spatial information.
You can simply touch the respective location or object on an interactive surface with your finger.
While speech is suited to provide semantically rich information,
for example, to specify an action that thas to be executed by the system.
Distributing interaction among these appropriate modalities has the potential to reduce Cognitive Load.
Despite this unique potential, there are practically no such interfaces in use today.
There are only few practical design guidelines and only very little user-centered research with functional interfaces.
So while understanding cognitive load in the context of MMIs is crucial, it is still under-studied.
To tackle this shortcoming,
we conducted a user study to investigate the Cognitive Load caused by different interfaces.
Therefore, we compared a multimodal speech and touch interface with a unimodal touch-only interface.
We specifically focused on Selection and System Control Tasks.
A Selection Task is simply a task where the user has to select a virtual object like the white mouse.
While in a System Control Task, the user has to instruct the system to perform a certain action,
for example, that the selected mouse should eat the cheese.
Selection and System Control Tasks are not exclusive to the domain of digital tabletop games,
but essential tasks that occure in many application domains.
We used the dual-task paradigm to objectivly assess cognitive load, since it is a well established tool in psychology.
A user has to perform a primary and a secondary task in parallel.
In our primary task the user has to command two virtual agents represented as mice to perform certain actions.
We implemented a fully functional speech and touch interface, depicted on the left
and a touch-only menu-based interface, depicted in the middle.
In the secondary task, the user has to move a tangible playing piece to a given target position as a reaction to an audio-visual cue.
If different versions of a primary task generate different levels of Cognitive Loads,
this is reflected in the performance of the secondary task.
The higher the Cognitive Load from the primary task, the worse the performance on the secondary task.
Here, you see the primary task with the unimodal interface on the right and the secondary task on the left.
Next, you see the multimodal interface.
Destroy this trap.
Collect this cheese.
Destroy this trap.
Collect this cheese.
We designed the interfaces to maximize comparability.
We made sure that the duration of an interaction is approximately the same for both interfaces
and consists of the same three phases:
Both interactions start by selecting a mouse via touch.
Then, you use speech to specify the action in the multimodal interface
while you have to choose the respective action from a menu in the unimodal interface.
The interaction concludes by touching the target position or object on the interactive surface.
Thus, the interfaces only differ in one aspect: the action selection phase,
where the interaction is distributed to a potentially more suitable modality.
In our study we used a two by two mixed design
with Interface as our within groups factor
and Task Difficulty as our between groups factor.
So, participants used both interfaces in either the high or low difficulty.
The difficulty level was set by the frequency in which secondary tasks occured.
In each condition participants familiarized with the task and interface in a short training session.
Afterwards, the participants performed the dual-task for three minutes
and finally answered our questionnaires before starting with the next condition.
In total we had 36 participants, so 18 per difficulty condition.
Our results confirmed our first hypothesis
that the multimodal interface causes less Cognitive Load then the unimodal interface.
Participants missed 58% less secondary tasks when using the MMI
and we found a medium effect of the perceived workload using the RAW TLX questionnaire.
Our results also confirm our second hypothesis,
that as a consequence of H1, the MMI is superior with regard to User Experience.
We found a high effect with regard to the perceived usefulness in terms of efficiency and effectiveness,
as measured by the pragmatic quality sub-scale of the AttrakDiff
and a medium effect with regard to the overall appeal of the interface,
measured by the attractivity sub-scale of the AttrakDiff.
Our third hypothesis states that
the difference between both interfaces maximizes with increasing task difficulty.
We found a significant interaction effect with regard to the dual-task performance.
If we look at the graph on the left here,
the x-axis shows the low and high difficulty condition,
while the y-axis depicts missed secondary tasks in percent.
We can clearly see that the difference in performance is significantly higher
in the high difficulty condition than it is in the low difficulty condition.
However, we found no significant interactions in all other measurements.
Thus, we only partially accept this hypothesis.
So in conclusion, our experiment showcases
how our multimodal interface causes less Cognitive Load
and achieves a better User Experience than the unimodal interface.
In addition, 32 of the 36 participants prefered using the multimodal interface, when asked.
But of course, one experiment does not solve the presented research gap.
We need more comparative studies to better generalize these results
and come up with practical design guidelines
to leverage the unique potential of MMIs in the future.
So, as a takeaway,
in our experiment MMIs proved to be promising.
Please consider using and researching them more often.
Thank you for your attention and have a nice day!