Hello and welcome everyone, my name is Johannes Zagermann and in this video recording I will present "It's in my other hand!"
Studying the Interplay of Interaction Techniques and Multi-Tablet Activities. This work was done at the University of Konstanz,
together with my dear colleagues Ulrike Pfeil, Philipp von Bauer, Daniel Fink and Harald Reiterer.
Let me start this presentation with showing you my current work. I guess it's nothing special just an ordinary workplace.
There are some papers, folders and pens lying around.
But in the middle of this workspace, there's my laptop.
I use the laptop for example to prepare presentations, write papers or for video conferencing, but
for some activities I also include tablets in my workflow.
I use tablets for example to read papers, annotate these papers with a pen or sometimes as an additional screen space.
In some cases also include other devices into my workflow, for example to take pictures of prototypes,
document my progress or if I need a break.
My workspace setup allows me to seamlessly incorporate various technologies,
different types of media and also even multiple modalities
in my daily work activities.
And with that I think this is a typical example of a workspace facilitated by the ideas of cross-device interaction.
Cross-device interaction especially with tablets is a quite popular topic in HCI.
Recent research has for example
investigated the effects and influences of a shared tablet as a shared workspace,
as an overview device for collaborative activities or
even how spatially-aware tablets can support collaborative sensemaking.
This research has emphasized the versatility of mobile devices such as tablets.
A recent study by Plank et al. looked into the topic of device utilization.
Which means they counted the number of tablets their participants used to complete the task at hand.
To their surprise they found out that most participants use only up to 2 tablets to complete the task,
which was in visual analytics tasks.
They referred is finding to legacy bias of only being used to work with single devices such as laptop
not taking the advantages or maybe just not knowing the advantages of
using multiple devices. In their paper, they also question if an extended training phase could alleviate
this legacy bias.
However, findings of a sensemaking task
using the Conductor system have shown
that participants easily included 8 or even up to 10
devices into their workflow. This might raise the question to what extent device or tablet utilization is based on a task
or the other way around to what extent the task can influence
the utilization of devices.
Similar to the discussed systems, previous research has also looked into
Cross-device interaction techniques and this research shows the variety of ways to transfer information
across devices and also shows the individual benefits or drawbacks.
These techniques include
traditional menu-like approaches,
portal-techniques allow you to drag and drop
content across the boundaries of a single device,
or techniques that are closer to real-world activities like picking up something and dropping it.
However, the comparison of those techniques across all this research is rather complex and as each study used their own
workflow or individual tasks.
Last year's cross device taxonomy
brings all of this together. They review application areas, enabling technologies, and also common interaction techniques for cross-device
interaction.
Additionally, they characterize common evaluation strategies.
So here it is worth mentioning that there are many different methods available,
but most of these evaluation methods are kind of disconnected from each other.
There are studies that evaluate
techniques regarding how usable they are
and on the other side there are papers than run rather informative studies, providing insights into users needs and help to understand their usage.
This separation of evaluation strategies inspired us to look beyond the boundaries of a single strategy and
Combine different but complementary angles of evaluation.
We did this by including different types of tasks into our study design
One task that allows us to study aspects of usability and a less restricted task
to build an understanding how participants work for example with a large ecology of devices.
This also allows us to see how different techniques are used across different activities and to study the influence of a task
for example on device utilization.
So this is what our study design looked like. We had two within subject factors: one was the task and the other one was the interaction technique.
So for the task, we had a predefined order of tasks: every participant started with task one and concluded with task two.
This way we could see the first task as a training and look into the learning effects
using for example a larger ecology of devices in task 2.
Secondly, we incorporated three different action techniques
We counterbalanced the order of them for the first task which allowed us to compare their usage and for the second task,
participants were free to choose their preferred technique,
for example to explore how to use the techniques and devices in a rather realistic scenario. Let's have a look at the tasks.
For the first task we used a abstract classification task. That was also used by previous research.
Here participants have to sort content. In our case,
there were 40 sticky notes with different labels and participants were asked to distribute matching sticky notes on different tablets.
Here we were able to look into classical usability metrics such as time or error.
As a counterpart, we included a complex sensemaking task that was also used by previous research. In the murder mystery,
participants are provided with hints and clues
written on sticky notes and they have to find for example the weapon, the time and place of the crime.
This allowed us to see how participants
handle a larger number of devices and just look into the effects of the training using the first task.
This is what our techniques looked like: the first one is called Menu,
The second one is called Tray and the last one is called Pick'n'Drop.
We chose these three techniques as do not require any additional engagement with device such as rotating or tilting
and they do not need any extra hardware augmentation such as a tracking camera.
Let's have a look at them:
The Menu button appears after pressing a sticky note and then you can paste it on another tablet.
The Tray is like a portal and you can just drag and drop
sticky notes into it and out of it. And Pick'n'Drop is related to picking and dropping
objects in the real world.
Here, you can see the setting of our user study:
Participants were sitting at an ordinary office table and they were provided in a stack of eight tablets.
For the study, we invited 24 participants and we had 40 sticky notes per task. Which means in task one,
there were different letters of the alphabet and in
the second task, there were hints and clues on the murder mystery.
With our study, we first looked into the performance of the three interaction techniques using the abstract classification task.
Then we were interested in the workflow of participants,
also using the abstract classification task and lastly we let participants solve the murder mystery and we explored their usage of techniques,
devices, their workflow, and also their opinions.
Let's start with some performance metrics:
So looking at the task completion times, participants needed statistically longer using the Menu technique.
Mainly due to the extra step to just show the Menu.
We analyzed the tablet utilization using the time-weighted tablet utilization score and here, a score of 1.0 would indicate that
participants used one device for the entire time and a score of 8.0 would indicate that participants use all tablets
for the entire session. Here, the score were rather equal: of all above
5.0
We also asked participants to rate techniques: Here, participants did not really prefer the Menu technique.
Only one participant said that the Menu is the preferred technique and 18 participants said that Menu is the least preferred technique.
Tray was preferred by 11 participants and 12 participants
preferred Pick'n'Drop.
Participants were only asked to sort and distribute the sticky notes for the classification task.
However, they were free in their behavior and workflow. So that's why we investigated how they proceeded.
We were able to characterize three different types of work behavior
The first one is called one tablet many piles and here participants focused heavily on sorting activities
they sorted all sticky notes on one tablet
and after that they started to distribute
those piles of sticky notes to another tablet.
The second group is called many tablets one pile each. Here, participants ordered the sticky notes by directly distributing them to
different
tablets and the major group was a rather mixed group. Here, participants sorted
the sticky notes until one pile was complete and then they distributed this pile to a new tablet
And started over with the next pile to sort and then distribute.
We also saw some interesting work behaviors during our video analysis:
So for example on the left hand side, you can see that one participant used both hands
to pick up a sticky note and another one to drop it on a different tablet.
On the right hand side, you can see a participant that did not hesitate to to stack all those tablets,
which was an indicator that they are finished.
With the murder mystery task, we allowed participants to choose their preferred technique.
And here the choice was related to their preferences for the first task.
The Menu technique was used less often than the other two techniques. Interestingly, 16 participants
combined multiple techniques for example to cut a sticky note with one technique and paste it with another.
Afterwards, we ask participants why they did use the techniques or why they didn't use it.
It was interesting to see the rationals:
Participants used Pick'n'Drop because it feels natural and you can use both hands.
But participants did not use Pick'n'Drop because they wanted to focus on the task. Which means that
the technique was also responsible for a certain task load.
Participants used tray as it was handy, but it was also used as a trash bin and
that's how it looked like:
This participant on the on the left moved all sticky notes with less important information into the Tray.
In the Tray, the sticky notes were a bit smaller and thus they needed less screen space.
On the right hand side. This participant had a dedicated trash tablet
Similarly to the other participant this one moved everything that was less important out of sight to focus on the task.
Focusing on the task was also important using Tray.
Some participants did not use it because they said the task was too complex to also incorporate a new technique.
Participants used Menu because it because it was fun and easy.
But it was not used because it took too many steps to complete the action.
We also calculated the time-weighted
tablet utilization score for the murder mystery.
Again, a score of 1.0 indicates the usage of a single tablet for the entire time and 8.0 would show the usage of all tablets
for the entire session.
Interestingly, we were able to calculate a score of
6.5 showing an even heavier usage of tablets as in the abstract classification task.
Now it's time to look back and reflect on our study design with the two tasks and three interaction techniques.
Using the combination of different types of tasks in a user study
is an effective way to gain different perspectives on a research problem.
Here using the first task as a training for the second task showed an interesting impact on device utilization and with that,
training can be seen as a method to cope with the mentioned legacy bias.
Future research should further investigate the effects and influences of suitable task combinations,
allowing complementary insights into participants' usage and needs of cross device systems.
This might eventually result in
components of an evaluation framework that allows us to compare techniques and systems and maybe creates a frame of reference for
future research.
If you're interested in more details,
tables, figures and numbers: Feel free to read the paper. Thank you and take care.