Hi, this is Difeng Yu, I am a PhD student from The University of Melbourne.
I am pleased to present our work gaze-supported 3D object manipulation in virtual reality. I
would like to thank my co-authors Xueshi, Rongkai, Hai-Ning from Xi’an Jiaotong-Liverpool University
and my advisors Tilman, Eduardo, and Jorge from The University of Melbourne
As one of the primary tasks in virtual reality systems,
object manipulation is used in many different application scenarios such as 3D modeling,
game development, and immersive data exploration. However, its primary input modality, which uses
virtual hands to “direct manipulate” an object has long been criticized for being
inefficient and imprecise, and likely to induce arm-fatigue in longer interaction scenarios.
In this research, we investigated different integration, coordination,
and transition strategies when incorporating gaze into current workflow based on mid-air hand input
for 3D object manipulation in VR. We examine a design space that considers
how gaze and hand input are integrated into different phases of the manipulation task,
how they coordinate with each other when starting the manipulation, meaning if the target
will snap to the hand position or remain in its original place, when starting the manipulation,
and how to transition from one modality to the other during manipulation,
either implicitly, or based on an explicit trigger like button pressing.
Based on this design space, we developed four gaze-supported manipulation techniques:
With GazeGrab, the gaze-indicated target snaps to the hand position once the selection
is confirmed. Next, the hand takes full control of the selected target during the
manipulation phase until the trigger is released. To manipulate an object through RemoteHand, a user
first points at it with eye gaze and then confirms the selection with a hand trigger. The target then
follows the rotation and translation of the hand, without snapping to the hand location.
3DMagicGaze establishes a safe region around the target once the initial eye-based selection is
confirmed. If the gaze point is within the safe region, only the hand can control the
transformation of the object. Otherwise, when the gaze point is outside of the safe region
and if the hand movement exceeds a threshold, the object snaps to the gaze point direction.
ImplicitGaze also forms a safe region around the target once the eye-based selection is confirmed.
If the gaze point is inside the safe region, hand input will control the object’s transformation.
Otherwise, if the gaze point is outside the safe region,
the object snaps to the gaze point direction.
We also introduced a mechanism, which resizes the safe region automatically
based on the user’s gaze behavior to prevent the gaze cursor from being “over-active”.
We evaluated the techniques through two user studies.
In the first study, we focused on the primary working space,
where all objects located within arm-reach distance and in front
of the user. We assessed the techniques in terms of user performance and experience.
In the second study, we further evaluated our techniques in a larger virtual environment with
distant objects and embedded the designed techniques into realistic workflows.
Our findings show that gaze might not offer significant performance benefits for transforming
objects in the primary working space but can be useful in a larger environment with distant
targets. Gaze input was also shown to mitigate the arm fatigue issue, and different integration,
coordination, and transition strategies can provide benefits for building more usable and
efficient object manipulation techniques in VR. We further summarize techniques' strengths, possible
applications, and provide potential compensation for their weaknesses. For example, we suggest
disabling the snapping function of GazeGrab when objects are within arm-reach distance and making
an oversized object transparent for implicit transition methods to reduce its intrusiveness.
Please refer to our paper for more details. Thank you.