Hi everyone, I am Zhen from the University of Toronto.
I am going to present our paper: “Armstrong: An Empirical Examination of Pointing at Non-Dominant
Arm-Anchored UIs in Virtual Reality”.
First, we thank Tovi Grossman and Michelle Annett for providing thoughtful feedback,
and Dan Clarke for recording voice-overs for our video figures.
Bimanual interactions are common in VR applications.
In a painting task, a stylus manipulated by the dominant hand often operates on a collection
of controls anchored to the non-dominant hand.
These apps utilized the space around our hands, and this is not surprising for physiologists,
since they have found that proprioception enables us to sense the position and movement
of our limbs through our sensory neurons, and “the presence of the hand prioritized
the space near the hand”.
Many applications have been treating the space around our hands as a reliable storage space,
and we might take it for granted.
But the form-factor of a smart watch still suffers from limited utility, and simple extensions
like these may not work.
Because they didn’t take the special locations into account, for example, most people wear
their watches on their wrist rather than on their elbow.
These observations motivated our work, and we ask ourselves:
“What areas around a user’s arm can be most quickly and accurately targeted?”,
and “What is a performance-optimized layout
of 3D UI controls around the arm?”
As a result, we conduct a user study to further investigate
where and how to place UI controls around the arm.
During the study, participants performed a series of pointing tasks in VR.
A start button was displayed on the right side of their body and could only be pressed
when they were in a “start posture”, determined by the virtual footprint on the ground and
the green bounding box.
The participant was instructed to select the target as quickly and accurately as possible.
More engineering details are available in our paper.
All possible target locations were distributed on a virtual ‘conical frustum’ grid
centered at their left arm.
Three independent variables were studied: longitude, latitude, and height.
Two awareness schemes were also included.
In the unknown scheme,
the participant would need to first identify the target before performing her selection,
and in the known scheme, the target location was visible prior to the trial start.
We used this condition to simulate users who have different levels of experience
using a particular app.
We adopted throughput (TP) as an ‘index of performance’ based on the actual
starting distance (D), size of the target (W), and movement time (MT).
Detailed stats analysis and full results from our interviews could be found in our paper.
Here I just share a selected list of results, and some discussions about them.
In terms of longitude, targets on the medial side of the arm had higher TP than those on
the lateral side of the arm.
TP decreased as targets moved up the arm along the latitude axis,
from the Wrist, to Forearm and Elbow.
TP also decreased as targets moved away from the skin along the height axis,
from Close, to Medium and Far.
As a result, when TP is of priority, the UI controls should be placed
on the medial side of the arm, around the wrist, and close to the skin.
Alternately, if UI controls are expected to be used infrequently or avoiding accidental
activation is critical, they could be deliberately placed in the ‘low-TP’ areas,
such as the lateral side of the arm, around the elbow, and far away from the skin.
We can also use the TP heatmap results as a reference to get the highest overall TP.
For example, if we wish to place 5 UI controls around our arm, we can select the 5 cells
from the heatmap with the highest TP value.
Furthermore, if more controls need to be added, the heatmap suggests a number of possible
axes along which to position controls while maintaining a high overall TP.
We also implemented a Unity plugin for these guidelines to apply our model automatically.
The designer can set up multiple preferences
and either preview the optimized layout in the Editor Mode,
or choose to tune the parameters and navigate the plans in the Play Mode.
Some interesting future projects could be built on this work.
First, future work could increase the number of levels of each variable to ensure that
more ‘continuous’ data can be obtained and used to interpolate between the levels.
Our plugin adopted a simple objective function to maximize the sum of TP to achieve the best performance.
Other parameters might be introduced to improve the algorithm, such as the density factor
and usage frequency of different UI controls.
Based on the TP, we could also dynamically resize the targets to enable a smooth and
consistent user experience over the entire space.
Thanks for listening!
For more details, please refer to our paper and video.