So now, let's move to our second presentation.
The authors will tell us about interesting solution that can combine
virtual reality players who have headset.
plus those who are just somewhere in a room and they will show us how
they can cooperate and collaborate in virtual reality.
So, the presentation title is spectator view: enabling a symmetric
interaction between HMD wearers
and spectaculars with a large display.
So, again, the screen is yours.
I'm Finn Welsford-Ackroyd
from Victoria University of Wellington, New Zealand
Toda,
I'll be talking about our paper titled spectator view, enabling
asymmetric interaction between HMD wearers and Spectators with a large
display.
The focus of our work was to enable communication and collaboration
between a user in VR and a spectator outside of the headset.
Head-Mounted Displays provide a highly immersive experience to a single
user, but they make it difficult to share that experience with
co-located spectators.
Typical solutions, like display mirroring, lack immersive
qualities.
And since Spectators can't be seen by the hmd user, communication is
hindered.
We wanted to create a bit of spectator system for VR applications,
targeting use cases which involve asymmetric collaboration like VR
demonstrations, or supervised VR training. In these cases, the hmd
where it is the primary user, but the spectator might want to assist
them in some way,
show off the experience to others or simply spectate and feel a sense
of presence.
The default solution for spectating VR is displaying mirroring where
the hmd's video feed is streamed to an external display.
This is easy to set up, but not ideal for the spectator.
I'm sure many of us have felt a bit queasy while watching a raw video
feed from an HMD.
At some point, the shakiness and the sudden pictures and roles can
cause simulator sickness.
Additionally, the lack of immersive qualities
make it difficult for the spectator to
understand what the HMD user feels.
Lastly, display mirroring provides no feedback mechanism for the viewer
to communicate with the HMD wearer.
An example of a paper with similar goals is ShareVR.
They used floor projection and mobile displays to present the
virtual world to users outside the headset.
They focused on giving non-HMD users feature, parity and specialized
collaborative applications.
Our goal was to create a more general solution for existing
applications.
Another example is ReverseCave.
They created a novel spectating solution, but didn't focus on
providing a means of collaboration or communication.
So our approach is, as follows.
The primary user utilizes
a traditional HMD with motion controllers.
The spectator views the virtual world on a large immersive display
and is also given to motion controllers.
One of these controllers can be used to orient
an independent virtual camera.
This allows a spectator to gain their own understanding of the virtual
environment and assist collaborative scenarios.
The spectators' other motion controller can be used as the laser
pointer which places a mark in the virtual world visible to the HMD
user.
This allows the two participants to communicate visually. The
large-scale immersive display was intended to provide a sense of
presence to the spectator, making their experience closer to that of
the HMD wearer.
We arrange the screens and a slightly curved ultra-wide layout and
increasing the spectators horizontal field of view to match the wide
angle perspective provided by the HMD.
We also applied a movement filter to the video feed shown on the
screen, which eliminates roll and smooth out the shaking movement to
reduce simulator sickness.
We provided the spectator with the ability to look around the scene
independently of the HMD user.
By default, the spectators' camera follows the HMD users' point of view.
This blue camera represents the HMD users' field of view, as seen from
above. and this orange camera shows what the spectator sees.
Without the spectator's input, the two cameras remain locked together,
just like display mirroring, but with the movement
filter applied to the spectators' view.
However, the spectator can use their trigger on that, the trigger
on their controller, to grab their camera and look around without
affecting the HMD user, like this.
When they let go of the trigger, their camera snaps back to following
the HMD user.
The spectator can also toggle free-look mode, which allows them to
completely decouple their camera from the hmd user.
So it remains in place when they release the trigger, this allows them
to look around using a "ratcheting" motion without needing to maintain
an uncomfortable hand position and that's what you can see the user
doing in this video.
Well, the spectator is free to manipulate the camera's orientation.
They cannot move to a different position from the HMD user in the
virtual world.
We considered implementing this but decided that it would be outside
the scope of a general-purpose spectating tool and it would not be
compatible with first-person single-player applications.
The spectator's other controller acts as a laser pointer into the
virtual world.
This was chosen as a method of visual communication with the HMD user
because it's a natural gesture that was familiar with from real life
interactions.
Using it straightforward, point the controller at an object on the
screen, pull the trigger and a glowing marker has placed in the
virtual world.
Implementing
the pointing tool was a little bit less straightforward.
In order to determine where the spectator is pointing,
we needed to track the position of the large display and relation to
the spectator's motion controller.
We achieve this by placing a Vive tracker on each corner of the
display and using those positions to generate a cylindrical hitbox.
We also needed to use the field of view of the spectator's virtual
camera to make sure that the cylinder was the right size.
When the spectator pulls the trigger, we fire a line trace from the
motion controller and record the collision point with a cylinder.
This gives us the position on the screen that the spectator is
pointing at, but we need to adjust for the fact that the screen stays
in place, while the spectator's camera can look in any direction.
We know that we can reach any point
displayed on the screen by firing a line trace out of the spectator's
camera.
We just need to figure out what angles are fired it at.
So we imagine the camera sits at the center of the cylindrical
hitbox, and calculate the angle as shown in the middle diagram.
This angle is then added to the spectator camera's forward vector.
and then we fire a line trace, which collides with the object at the
the spectator's pointing at.
And that's when we place the marker, the advantage of using a
cylindrical hitbox
is that even when the user misses the screen, the market is still
placed at the corresponding position in the virtual world?
The same concept could also be applied to support Dome displays.
If the cylinder was converted to a sphere.
To analyze our system,
we performed a user study.
We focused on evaluating the impact of camera control and the pointing
tool as we consider these to be our most important contributions.
We had our users complete a collaborative task, then use
questionnaires to evaluate the usefulness of each tool and the effect
on the user's experience.
In the task, the virtual environment was a square room with a large
grid on one wall.
The other walls hold 40 unique moving cubes with various colors and
patterns.
The HMD user can pick up these cubes and place them in the grid.
The spectator was provided with a printed screenshot of the
grid with five cubes placed in it as shown in the image on the right.
The participants must communicate and collaborate to find the cube,
displayed in the screenshot and place them in the correct layout.
In this video,
you can see that the orange marker is the spectator showing the HMD
user which cube to pick up and where to place it in the grid.
We had four conditions in the trials.
We had the baseline, which was essentially the same as display
mirroring, but on a large display,
with the motion filter enabled, we had the pointer tool only
camera control only,
and then both tools enabled with a spectator was given two motion
controllers to use both tools at the same time.
We had all 24
participants complete a trial for each condition in each role.
So they each got to be a spectator and an HMD user.
The same eight grids
we used in a random order for each pair of participants.
And if you're wondering what the strange motion controls are in this
photo.
Those are the spectator's controllers.
They were made of Vive
Trackers tape, to PlayStation move controllers, to get around the
limit of two controllers per headset, and the HTC Vive Pro.
After each trial, likert-scale questionnaires were given to both
participants.
We gathered information about the usefulness and usability of each
tool.
Each user's perceived presence in the virtual environment and some
specific metrics of social presence, including attentional allocation,
and perceived message understanding
These allowed us to see where the participants were able to focus on
their partners actions and understand what they were doing.
We found statistically significant results for perceived message
understanding when the pointer tool was enabled.
Users in both roles felt that the spectator's thoughts were clearer to
the HMD user.
As for attentional allocation, both users felt the spectator
did not remain focused on the HMD user when they had the camera tool
available to them.
This was expected, but it does highlight a potential downside of
giving more independence to a spectator.
When the pointer and camera tools are available,
spectators founder easier to complete the task and they felt more
present in the virtual environment.
Additionally, users in both roles
found the decoupled
spectator camera useful, especially when paired with the pointer
tool.
We did not find statistically significant results with regards
to trial completion time by performing pairwise comparisons.
We do note that trials were completed 13 percent faster on average
when the pointer tool is available.
So while we didn't see a great leap and efficiency, the subjective
results from the questionnaires highlight the benefits of our system.
In conclusion, the contributions are a novel approach
for a symmetric collaboration between an HMD user and a spectator, a
user study that evaluates the effectiveness of our approach and a
reference implementation covering display arrangements,
spectacular controllers, user interface and motion filter for the virtual
camera.
For future work, we could apply the system to other VR applications
and see how useful it is and different use cases.
We could also investigate extending the system to include more than
one active spectator.
Thank you so much for listening.
Feel free to ask questions or send me an email.
Okay.
Thank you very much for your interesting presentation.
So do we have any questions from the audience?
If so, I will start with my question.
I guess that participants,
I mean spectator only see the results of activity of the HMD user.
Also an HMD the user only see the results of the activity of the
spectator, where you also considering showing the position of both of
them in virtual reality.
So, for our system, they are always locked in the same position, but
they can look in different directions.
something
I didn't mention in
the presentation is that we had a compass UI element, which was meant
to, was like a 3D circle with an arrow pointing where your partner is
looking relative to where you're looking.
But yeah, our users
in the test find it very useful.
So, definitely, something that we could improve upon.
So they were, they were aware of each other
not only seen the results of the activity,
I mean, that that pointer area.
where is the pointer,
Okay.
Thank you very much for your answer. It's
clarified now.
Cool.
Thanks.
Do we have more questions?
Do you have any questions on chat?
Okay.
So, again, thank you very much for your interesting presentation and
again, pick up walls for you.
Thank you.