Hello everyone, I’m Nianlong Li from Institute of Software, Chinese Academy of Sciences.
The topic of my report is “get a grip evaluating grip gestures for VR Input using a Lightweight Pen”
Virtual Reality is emerging as the next generation computing platform, for a diverse set of domains,
including data analysis, artistic creation, and clinical settings
All of these require high precision input.
Current design of handheld controllers, such as the HTC Vive and Oculus Rift, typically
include haptic and touch-sensitive controls, allowing users, to launch continuous and discrete
commands or movements, in the virtual environment. However, these controllers primarily exploit
wrist rotation gestures, missing the opportunity to fully utilize the fingers’ capability.
In our life, it is an inherent ability to use the flexibility of fingers to operate tools.
Such as eating with chopsticks, throwing darts, and writing with a pen
When using pen as an input device, it can greatly utilize the flexibility of fingers
while ensuring high precision input.
But we possess limited knowledge as to what extent pen controllers can benefit VR target
pointing, selecting, and gesturing.
To this end, we look into the potential of a pen-style controller and grip, that can
exploit the fingers’ rich dexterity, for high precision input in VR.
And we conducted two studies, to understand the use of a pen, as an input controller in VR.
The aim of the first study was to examine the effect of pen grip style, and to identify
the best suited grip posture for a pen, when trying to achieve the largest comfortable
range of motion.
Based on prior work, as well as those commonly used in art, we assessed a total of five grip postures.
They are tripod grip at the front end of the shaft, tripod grip at the rear end of the shaft,
quadropod at the rear end of the shaft, pinch and overhand.
We used a 3D-printed pen with four reflective markers, which were tracked by OptiTrack.
We recorded the pen tip’s spatial position, and the pen’s orientation information at
every frame. A with-in subject design was used, and the
Factors include five grip postures and eight directions.
In total we have four thousand three hundred
and twenty trials.
We calculated the euclidean distance, travelled by the pen tip, and the
angle tilted by the pen shaft.
The participants were asked to complete a NASA-TLX form to rate workload factors from
one to seven after the study.
This is the process of study 1.
Participants were prompted to tilt the pen, with a particular grip posture in eight directions,
as far as they can, without reaching discomfort.
Results showed that the moving distance with Tripod Rear End posture, was significantly
larger, than all others.
The Tripod Rear End posture, led to a significantly larger tilt angle, except for Quadropod Rear End.
Across postures, the moving distance in the Southwest direction was the largest, and the
shortest distance was in the East direction.
The tilt angle in the West direction was significantly larger than most of the directions, while
the East direction, let to a significantly smallest tilt angle except for the Southeast.
This is in line with our expectations, as all of participants were right-handed, with
their actions appearing to be more flexible, when tilting the wrist and fingers inwards,
rather than outwards
The NASA ratings showed that all grip postures, were less than three point five, except for
Quadropod Rear End , some participants said that they had no experience in writing with
Chinese brush.
It’s hard for them to use the Quadropod Rear End skiifully.
The Tripod Rear End posture was rated as significantly easier to use than others
However, it is unclear how this grip posture would benefit typical VR interface tasks,
such as target selection.
Then we conducted sudy 2 to examine it.
In this study, we aim to understand how efficient pen input, is for VR target selection, in
comparison with the typical Palm Grip, used in current commercial controllers.
The study consists of two sessions, corresponding to two common interface gestures, Poke and
Tilt, for menu and object selection tasks in commercial VR devices.
Poke gestures are used to select items on menus that are physically reachable.
Tilt gestures are often used alongside ray-casting for distant pointing.
In each session, we used the within-subject design, the factors included two grip postures,
Pen Grip and Palm Grip.
Pen Grip refers to the Tripod Rear End grip posture in study 1, and Palm Grip refers to
the grip posture used in current commercial VR controllers.
We set two levels of target width, and three levels of target distance.
This yielded six distinct Fitts IDs.
The selection time and error rates for poke were recorded.
Participants were asked to complete a NASA-TLX form after the study.
In poke session, the action is identified by the pen entering the 3D target volume,
and then exiting it.
In tilt session, we used the dwell trigger mechanism, the participants need to hold for
500 milliseconds to confirm the selection
The results showed that the pen grip posture improved the target selection time with both
Poke and Tilt gestures.
The error rates of Pen Grip was significantly lower than Palm Grip in the Poke condition.
Fitt’s law fitting well for tilt gesture, with increasing IDs, the Pen Grip selection
is more efficient than Palm Grip.
On the other hand, we did not see a strong correlation with Poke gestures, especially
with Palm Grip.
This might because the pen overshoots the targets, as there was no physical surface
that provides haptic feedback, when using the Poke gestures.
The average NASA-TLX ratings for all grip postures were less than three point five,
and there was no significant effect between the two postures in Poke and Title.
Though Pen Grip was faster in acquiring the target, some participants said, that it did
raise the fatigue issue.
Then we present three scenarios enabled, by VR pen input, and grip postures.
It is easy to switch between the Pen Grip and Palm Grip postures.
This enables new interaction opportunities.
For example, one can select furniture with the Palm Grip, and switch to the Pen Grip,
to invoke a menu, and perform operations, like picking colors or adjusting scale values.
With the pen grip, one can easily perform short and rapid input bursts, such as tilting
and poking.
These gestures can be used to trigger, scrolling or zooming operations, on a web view by tilting
and poking the pen, respectively.
Virtual pointing techniques whose tool origin is located at the hand position, can suffer
from the eye-hand visibility mismatch.
The Pen Grip posture avoids this problem, as it keeps a high consistency, between the
visibility of eyes and that of hands.
We can see that there is a good potential, to use the pen for VR operations in the future.
Nonetheless, our paper has several limitations, some of which are worth exploration in future research.
Besides tilting and poking gestures, the rolling gesture is also a viable input method in VR.
One challenge of performing pen rolling gestures in air, is to identify suitable grip gestures
that keep a good balance of range of motion and stabilization.
In our second study, dwell was used to confirm a selection with tilt gestures and ray-casting.
Designers could also add a physical button to the pen, but it raises a new challenge
that it is unclear how switching the fingers’ roles frequently would affect performance
in pointing tasks.
Similar to most controllers, haptic feedback is critical for creating successful use experiences
of the pen.
Adding vibrotactile feedback to the pen device is feasible with a high frequency linear actuator.
Despite the technical challenges, developers should aim for VR pen’s form factor; larger,
heavier form factors may play a critical role in the user’s experience.
in conclusion, We look into the potential of a pen-style controller and grip that can
exploit our fingers’ rich dexterity for VR input
We found the tripod grip at the rear end is an optimal grip posture for a pen when trying
to achieve the largest comfortable range of motion.
This improved the target selection time with either poke or tilt gestures, and was less
error-prone with poke gestures.
We discussed potential interaction design and interaction opportunities with pen input in VR
Thank you for listening, if you have any questions, please contact my e-mail.