Hello, my name is Pascal Jansen and I‘m presenting Share: Enabling Co-Located Asymmetric
Multi-User Interaction for Augmented Reality Head-Mounted Displays.
We designed ShARe with the idea of enabling Non-HMD Users to become part of the virtual
environment of the HMD User and enable them to interact and explore the augmented content
together.
Due to recent progress in consumer technology, such as higher display resolution and miniaturization,
Augmented Reality and Virtual Reality is getting dominantly realized using head-mounted displays.
While AR and VR technologies are often treated separately,...
... they share common problems, which arise from near-eye display usage, such as isolation
and exclusion.
Isolation occurs when an HMD fully occludes the real world from users and thereby isolates
them from the environment.
For example, here the HMD user is isolated from the group while interacting within his
own world.
Exclusion occurs whenever information is only visible to the HMD user and every user in
the environment without access to the HMD is excluded from the experience.
This example shows multiple excluded bystanders who do not know and see what the HMD user
is doing right now.
There are approaches that fight the exclusion for VR, such as CAVE-like systems and Collaborative
Virtual Environments.
Gugenheimer et al. created a system for VR that prevents isolation of HMD users and includes
Non-HMD users in the experience. Similar to our idea, they used a projector
setup to visualize the virtual world to the outside.
While this and prior work is already addressing the issue of isolation and exclusion for VR
HMDs, there is currently a lack of work focusing on these factors for AR HMDs.
So, how can we include people in the environment without an AR HMD and enable them to understand
and interact with the virtual world of the AR HMD user?
So we built ShARe, which is a modified HoloLens AR HMD, a DLP projector, and a servo motor,
which is controlled by an Arduino Nano.
Using the internal cameras of the HoloLens, we allow Non-HMD Users to interact with the
projection via touch input.
This enables a new form of asymmetrical interaction, in which the HMD User can share similar content
or combine both technologies to create a new and more inclusive type of display.
Without corrections, the projection is distorted and does not match the virtual content.
The distortion correction aligns projected content with the virtual world of the HMD
user. We used a servo motor attached to the projector
to compensate head movement and to ensure alignment.
Various Use Scenarios arise from our flexible hardware setup, including different use modes
(such as, sitting, standing, or walking) and usage of any planar surface in the environment.
For example, sitting at a table with a group of friends and playing a dice game.
Standing in front of a whiteboard with colleagues and placing notes.
Navigating with a virtual floor plan while walking.
Or, playing air hockey on a clipboard if there is no suitable surface around.
To explore the design space we implemented three applications: Labyrinth, Treasure Hunt,
and Floor Planer.
The goal of the collaborative game Labyrinth is to influence the character’s path by
shifting the labyrinth tiles. The HMD user must guide the non-HMD user.
Treasure Hunt is a turn-based competitive board game in a pirate setting.
The HMD user places treasure chests on the game board fields that must be found and marked
with flags by the non-HMD user. Both users can use special items from a store
that have different effects on the gameplay and the surrounding fields.
Floor Planner is an application for architectural design, which allows both users to view and
modify floors, rooms, and objects.
While developing and testing ShARe, we discovered various challenges specific to the combination
of AR HMD and projector.
The HMD user may perceive a mixture of inside and outside visualization when the augmentation
virtually overlays the projection. A solution for this is to show all content
using one of the visualizations, for example, the projection.
Another solution is to show specific objects that are relevant for the user in each visualization.
Due to intentional or unintentional head movements, the projected image is constantly shaking.
We applied a filter to minimize jitter and to smooth the movement.
If an orthographic projection is used, less object information and details can be displayed
in the outside visualization. The perspective projection allows us to convey
more object details from different perspectives. By placing the marker on the forehead, the
non-HMD user can set the perspective camera position.
To conclude, ShARe is a proof-of-concept prototype using an AR HMD with a servo-controlled projector
to visualize the virtual world for Non-HMD Users.
They are able to interact with the HMD User and become part of the experience.