I’m Joanna Bergström, and together with Tor-Salve Dalsgaard, Jason Alexander, Kasper Hornbæk,
we reviewed 20 Years of Virtual Reality Studies of Object Selection and Manipulation.
Designing experiments for evaluating interaction is tricky,
even when the focus is in the basic interactions like object selection and manipulation.
We have some standards and common practices for experiment tasks on object selection,
such as these reciprocal task, disc transfer
task, and pin transfer task from Fitts’s studies almost 70 years ago.
And we have the ISO 9241 part 9 standard for tapping tasks in 2D.
However, in VR these standards are harder to apply.
In VR, the objects of interest are not limited to a 2D display with its frames in front of us.
And even if researchers have adapted the 2D iso task into 3D layouts
like we did in this spiral-shaped layout following Qian's and Teather's example, in VR the objects of
interest can also be all around the space and around ourselves, and occluded by each other.
To find out how should we evaluate object selection and manipulation in VR,
we reviewed 20 years of studies on that.
Our aim was to analyse the studies to form an empirical basis of best practices in VR studies.
We reviewed studies of object selection and manipulation in VR published between 2000-2019
in the central conferences and journals in the field: IEEE VR, VR journal, VRST, CHI, and UIST.
The 39 papers included in our sample consisted of 48 studies,
and out of these 30 studies evaluated object selection, and 18 object manipulation.
One of our key findings was that such as seemingly mundane decision
as laying out targets for selection task is indeed difficult in the context of VR.
The strategies studies used included, for instance:
Target pairs like in the Fitts’s
reciprocal tapping task, but placed, for instance, on frontal and lateral axis.
2D layouts following the ISO standard layout,
adapted for instance in a horizontal form here on the left and a vertical form on the right.
And 3D layouts with a random arrangement of targets.
The studies further vary in the use of distractor targets, occluding targets,
shape and size of the targets, and many other factors of the experiment design beyond the task.
Our aim with this review was to help researchers,
including ourselves, to design and conduct VR studies of object selection and manipulation.
To begin moving from our findings toward common practices in VR studies,
we synthesise our key observations in:
10 recommendations for designing studies.
And a checklist to support the use of recommendations and making reproducible studies.
We also discuss research directions for VR study methods.
In particular, we find a need for tools that help
to accumulate knowledge and findings of object selection and manipulation techniques.
In our sample, 64.6% of the studies used a completely new task
rather than building on existing studies, and therefore
comparison across those or meta-analyses of findings are currently difficult.
Aiding tools for this would be for instance, test suites and reference tasks.
We hope to continue working on these first versions of the recommendations
and the checklist on VR studies with you at http://vrevaluation.github.io.