Hello everyone. My name is Raphael James and I am here to present:
Evaluating the Extension of Wall Displays with AR for Collaborative Work.
This work was done with Anastasia Bezerianos and Olivier Chapuis from Université Paris-Saclay.
Wall displays can accommodate multiple people viewing the wall from a close distance and
from afar. With their size and high resolution, they are used to display a large amount of data.
But when there are too many elements, even wall displays run out of space eventually,
and this space is hard to add. Indeed wall displays require a large space to be placed in
because they require a mounting structure and a complex setup.
This complexity makes it costly and time-consuming to change their
shape or increase their size by adding screens.
Previous work increased the size of desktops, smartphones, and Cave with AR,
so it seemed natural that we use AR to increase the shared workspace available on wall displays
and the offload content when it becomes too much for the wall to handle.
Offloading the content into AR changes greatly the space available to the user,
but what else does it change for the users what is the cost of such an addition?
Previous work studies the use of space whether when working on the wall display or in AR or VR.
As this past work does not consider a physical screen and AR content combined, we cannot rely on
previous insights but we can get inspired by them. So we decided to use sensemaking tasks,
like previous studies, to evaluate space usage. During sensemaking tasks, users need to be able
to move documents around to create semantic groups and explore the data.
So as a first step for this work, we needed to create a layout management
system where AR HMD augments the wall size display to offload this extra content.
For this, we created a system based on three components. I will show the
physical elements in black and the AR elements in color in the small picture.
First, we use boundaries. Boundaries delimit the environment of the users.
They represent the space where items can be placed
and boundaries are not visible in AR but are shown here for explanation purposes.
Second, we define surfaces. Surfaces represent the workspaces used by users. They take the
shape of traditional desktop-like windows and are shared by all users. The shared
surfaces are placed by users on the boundaries. Some special surfaces exist in the system, like
the actual wall and personal species that we can observe around users, that are entirely personal.
Third cards. Cards are items that can be placed by users on different surfaces. They represent
text documents, images, visualizations, or other content providing information.
We allow users to create surfaces and place them around the room.
And we designed different techniques to move items around. A user can move either
cards or surfaces by a drag and drop using a clicker. To ease the movement of multiple cards,
we allow the selection of multiple cards, either one at a time or many at once.
Once selected, cards can be moved in a group between all existing surfaces.
A personal space exists to store cards, accessible near the user like a belt. The user only needs to
look down to see the stored cards. The stored content can be placed after on surfaces again.
To observe the cost of adding AR to the wall to offload content, we ran a user study.
Our experiment had two conditions: the first one is our system that we compare for reference
to a wall-only system that uses the same techniques for the movement of items.
And we ran this experiment with 12 pairs of participants.
We use two basic sensemaking tasks that require layout management.
First, the classification task, where participants were asked to
classify cards between three colors without leaving any cards aside.
This task is inspired by other classification tasks used in previous works on Wall display.
Second, the storytelling task where participants were asked to
create a story using 10 cards of their choosing.
We chose these tasks to promote two cooperation behaviors.
First, a loose collaboration for the classification task, where the work can
be done in a divide-and-conquer strategy. And second, a tight collaboration
for the storytelling task where both participants must agree on every step.
All groups did the classification task and then the storytelling task for each setup condition.
For this experiment, we use two Dixit datasets, counterbalanced across conditions.
The wall is completely covered at the start of each task for each condition,
to simulate the crowded wall situation, as you can see here.
Both data sets are composed of colorful and abstract images to
promote discussions within pairs of participants.
Now that we have a system to test the use of space, we want to answer four main questions.
First, is the extension of a wall display environment useful? And when is it used?
Second, does the addition of AR affect collaboration?
Third, what is the cost of adding AR? And fourth, how is the AR space used?
To answer those questions we need to study and
analyze the collaboration sessions of participants working together.
As videos cannot show what happens in AR we developed a replayer tool.
By giving it the log of a session, this tool allows us to re-watch what this participant
did on the wall display and in the AR space. It can play a session at different speeds and
from different points of view, including the point of view of each of the participants.
After observing the participants' collaboration sessions we saw
some impact of AR in specific cases. We did not observe differences in collaboration
strategies during the story task, where for both conditions, participants worked closely together.
But we observed that the addition of AR impacted strategies for loose collaboration tasks.
Indeed during the classification task, in the wall condition,
groups displayed four different strategies, from dividing the work among themselves to
working tightly together on each color. For the AR condition, all pairs used AR
surfaces to classify one color and divided the task among themselves.
Also, AR shows no cost on performance.
We see no significant difference in the time to complete the classification task.
This means that even with extra actions to create surfaces, and the large distance to
move cards across them, this did not have an impact on the overall completion of the task.
For the story task, there is no significant time difference either, which is less important,
as by design, discussions between users take more time than manipulation in this task.
What is interesting when considering adding AR
to existing surfaces is that AR is preferred overall by participants.
They enjoy it more than the wall condition and felt more successful with it.
While for both conditions you were wearing it sets, users ranked the wall less
physically and mentally demanding than the AR. But we feel that these last two results may be
related. It is possible that participants walked around more, and focused more in the
Wall+AR condition because they were more engaged. And this relates to the fact that they preferred
the AR condition and felt more successful with it. Of course, this is only a hypothesis.
And finally, we saw that AR modifies the use of space.
We saw participants dividing the wall into zones for the classification task
for the wall condition. Like for example, here, we have pictures of
the results from the participants' tasks. You can see on the top images that pairs
divided the wall into three groups of cards, or into three lines of cards.
But in the AR condition, they used the AR surfaces as their primary interaction workspace. They
created AR surfaces, usually positioned on each side of the wall, to organize one color on the
wall display and two on the virtual surfaces. Some pairs also decided to place every category
in the AR space, by creating one surface per color and removing all cards from the wall.
For the story task, for the wall condition, all participants took time making space on
the wall to arrange a story in the created space, as you can see here on the left.
And on the right, for the AR condition, we see that participants removed the
story cards from the wall to place them directly on the AR surface on the right.
Similarly to the classification task, participants in the AR condition use the AR surfaces as
a primary interaction workspace, promoting them to a first class interaction workspace.
Overall this work showed that we can create an extension of the wall in AR,
that is used when the wall does not have any more space and improves user experience.
But also that this AR extension has an impact on collaboration strategies. More
uniform strategies were used in the AR condition compared to the wall alone.
But we also observe that AR did not show an impact on performance while it's more
demanding than using a wall display. And finally, users turned quickly
toward virtual space to layout their content. So much so, in fact, that during the story task,
all but one group presented their story on AR surfaces instead of the wall display.
Today's takeaway note is that when used with the wall,
AR surfaces were promoted to primary interaction workspace by users.
This was Evaluating the Extension of Wall Displays
with a AR for Collaborative Work. Thank you for listening.