Hello my name is Robbe. Today I will be presenting SelectVisAR:
Selective Visualisation of Virtual Environments in Augmented Reality.
Which is the work of my colleague Jihae and myself,
under the supervision of prof. Adalberto Simeone. This work is concerned with Cross-Reality
Interactions. Which are interactions between users in different realities.
We often describe these realities through Milgram's Reality-Virtuality continuum.
Which ranges from the real environment to the virtual environment and realities in between.
For example, VR will be situated on the right of this spectrum.
While AR is located more towards the center, being a combination of both real and virtual elements.
And then the physical reality is all the way to the left,
and users in the physical reality are often referred to as external or non-immersed users.
The interaction between non-immersed users and VR users has been researched before.
For example in ShareVR, which presents a system involving a spectator screen, a floor projection
and a handheld monitor to enable non-immersed users to interact with VR users.
This system was successfully used to implement a variety of games.
Another approach is presented in FaceDisplay. In this work a number of
small screen were mounted to the head-mounted display of the VR user, allowing the external
users to see into the Virtual Environment. External users were also able to interact
with the Virtual Environment through mid-air gestures or through touchscreen interactions.
We are concerned with interactions between VR and AR users.
Which is an area that has been researched less extensively before.
In "Assymetric Collaborative Interactions" Symmetric and
Asymmetric interactions were compared. For example, symmetric interactions
involving either two VR, or two AR users, compared to the Asymetric interaction
involving one VR and one AR user. The task that was investigated
was the manipulation, collaborative manipulation, of small 3D objects.
Another work in ObserVAR which presents a
system in which a teacher using AR can visualise a group of students, who are using VR.
Different ways of presenting this visualisation were presented, for example "First Person View",
"World in Miniature" and "World Scale". "First Person View" and "World in Miniature"
were displayed above the VR user's avatar, whereas "World Scale" scaled the size of the
Virtual Environment to the size of the physical environment, which was available to the AR user.
In our study we will investigate how to visualise a VR user and their Virtual Environment to an AR
user, while providing enough information for the AR user to understand what the VR
user is doing, while also not providing too much information
so that the AR user can have a clear view of the real environment.
We will do this by investigating different ways of filtering this information. Which can be dynamic,
changing over time, or a static selection of objects.
We will also investigate different scales of this visualisation, for example
room-scale or a scaled down view. Before answering these questions let's
first take a look at the VR simulation we used as an example in our study.
We implemented a VR task which involved the VR user being a bartender, where they could
manipulate virtual objects such as fruits, bottles and glasses,
in order to create a cocktail consisting of a simplified three-ingredient recipe.
Now the question remains of how to visualise this task to the AR user,
who is colocated with the VR user. For this we implemented four different techniques.
On room-scale we have "Everything", "Proximity" and "Interactive" and on the 1 in 5 scale
we have the "Dollhouse". "Everything" is symmetrical
between VR and AR users, with them being able to see the same parts of the Virtual Environment.
"Proximity" is a cylindrical filter which follows the VR user around,
so that the AR user can always see the area of the Virtual Environment which is close to the VR user.
"Interactive" is a static predetermined selection of interactive objects which we considered to be
a minimal amount of information for the AR user to still be able to understand
the actions that the VR user is performing. And finally there is "Dollhouse" which is a
1 in 5 scale model of the Virtual Environment, in which the AR user can only see the VR user's avatar,
while the physical VR user is standing off to the side of the visualisation.
Then we evaluated these four techniques in a user study.
Participants first filled in a demographics form and had an introduction to the Hololens 2.
Then the researcher performed the VR bartender task, which had a number of extra events
scripted in, such as dropping a piece of fruit or moving a bottle without pouring.
The AR user was then in structed to look out for these events,
so that they could indicate them after the observation was complete.
This way we could analyse how well they were able to observe the VR user's actions.
They also filled in, after each trial, a SUS presence questionnaire
and a Similator Sickness Questionnaire. The study followed a within-subjects design.
With each technique being presented to each participant, in a counterbalanced
order, for a total of four trials. After the study participants could
indicate a preference of which techniques they found most useful in completing their task. As
well as they were interviewed on why they found certain techniques more useful than others.
A total of 13 participants participated in this study. The ranking they were able to give
at the end indicated that they preferred "Interactive" and "Proximity" the least.
From the interviews it became clear that for proximity
the lack of awareness of the invisible part of the Virtual Environment was the reason for this.
And for "Interactive" they missed the bar counter as a point of reference in the scene,
and they were also very distracted by the fact that there were floating objects.
Event recognition was analysed by calculating a competence score, which is the true positive rate
minus the false positive rate, where 1 is a perfect score which indicates that
the participant indicated the correct events, and only the correct events.
There wasn't any significant difference between the techniques, with "Interactive"
scoring the highest, followed by "Proximity", "Everything" and "Dollhouse" scoring the lowest.
Based on this feedback we designed two follow-up techniques. First is
"Spotlight" which replaces "Proximity", and "Context" which replaces "Interactive".
"Spotlight" addressed the need to remain aware of the out-of-focus part of the environment
by representing this part of the environment as outlines.
This way the users could stay aware of the out-of-focus part of the environment
without it hindering their view of the real world.
"Context" improves on the need for more context that participants had with "Interactive" by
adding the supporting furniture to the interactive objects, which are the bar counter and the sink.
These techniques were then evaluated in a study with a similar procedure as the first study,
alongside "Everything" and "Dollhouse". We had another 13 participants,
and this time "Spotlight" was preferred the least of all techniques.
Participants cited the lack of control over the visualisation as the reason for this,
as they didn't like being able to decide for themselves which part
of the environment they were able to see. We also found a minimal difference between
"Context" and "Everything". Not only in preference but also in the interviews where
participants said that they also experienced the techniques as being very similar, and
some participants even expressed that they were not able to tell the difference between them.
We analysed event recognition in the same way as in the first study. This time there
also was no significant difference, with "Context" scoring the highest followed by
"Everything" and "Spotlight" with "Dollhouse" again scoring the lowest.
On the effect of selection we found that "Everything" and "Context are
experienced as being very similar. And that dynamic selection was not
preferred, in our case "Proximity" and "Spotlight", because of the lack of
control participants had over this visualisation. Then on the event recognition task no significant
differences were found between any of these selections with the values being about the same.
On the effect of scale we found that "Dollhouse" provided users with a better overview.
But also that sometimes users found it to be too small, which is also supported by the "Dollhouse"
scoring the lowest on the event recognition task in both the studies, albeit not significantly.
So in conclusion, if an overview is preferred we recommend using
a scaled-down "Dollhouse" perspective.
Users generally prefer static over dynamic selection.
And we found a good static selection in our context technique, which included interactive
objects and their immediate context. However, further research is required to
generalise this approach to different environments.
Thank you for listening to this presentation on SelectVisAR:
Selective Visualisation of Virtual Environments in Augmented Reality.