Good morning, good afternoon, and good evening. I'm Myung Jin Kim from the MAKinteract Lab in KAIST,
and I'm grateful to present to you our work "SpinOcchio: Understanding Haptic-Visual
Congruency of Skin-Slip in VR with a Dynamic Grip Controller," on behalf of my amazing co-authors
Neung Ryu and Wooje Chang from KAIST, Michel Pahud and Mike Sinclair from Microsoft Research,
and my advisor Andrea Bianchi from KAIST. Let's dive right in. Now imagine yourself inside a VR
cooking game trying out challenging recipes, mixing interesting ingredients, and preparing a nice meal.
You reach out into the refrigerator and grab a bottle of soda to serve with the meal, and right as
you're about to pop open the bottle cap, you look at the bottle in your hand and wonder, "hmm...wait
a minute. How is this supposed to feel? And what would I need to make it feel real? What would I
need to be able to feel the contours of the bottle as it slips through my fingertips?" Well, intuitively
you would need some tactile feeling of a firm surface slipping against your fingertips in a certain
direction, as shown here with red arrows, while the object that is held moves in the direction
of the green arrows. Now similarly, if the object were to spin inside of the hand as shown here,
the fingertips would be experiencing slipping in opposite directions. So how do we achieve
this? Looking at prior work, haptic feedback using slipping surfaces, also called skin slip haptics,
was achieved using rotating cylinders, spheres, or moving belts that slip against a fingertip.
As prior setups did not support multi-finger skin slip, we came up with our own approach using flat
spinning discs. Additionally three recent works have used skin-slip in VR for one fingertip.
Our work differs from prior work, focusing on enabling multi-finger skin slip interactions in VR.
With the disc, to enable fingertip skin-slip in any direction tangent to the fingertip, we came
up with the following approach: When you touch any point on a spinning disc away from its center,
your fingertip is pulled in a direction perpendicular to its position from the center.
Now if you touch different locations around the disc, the skin of the fingertip slips in different
directions accordingly. Now instead of moving the finger, if we pivot the disc around the fingertip,
the same range of skin-slip directions can be achieved. With this working principle, and after
a few iterations, we built the current version of SpinOcchio. A pivoting disc mechanism module
was built and placed back to back to allow skin-slip feedback on both the thumb and index fingers.
The distance between each module was also controllable to enable different grip widths
to reflect virtual objects of different thickness. Here is the mechanism in action.
The disc can continuously pivot around the placement of the fingertip in a range of 180 degrees, while
spinning in a clockwise or counterclockwise direction. The distance between each fingertip
ranges from a minimum of 26 millimeters to as much as one's fingers can spread apart.
Now as skin-slip direction perception has not been investigated in prior work,
using this mechanism, we first conducted a haptic perception study with 12 participants and found
the angle perception results to be comparable with prior work. Also, generally users perceive
better the skin-slip directions using only one finger than using both, as shown by the JND values.
With this basic understanding, we conducted a second study with 12 participants in VR
with visuals of virtual objects and tested how the haptic feedback was perceived in relation to
the visuals when involving the following three variables: First, the proprioceptive cues from
passive and active hand motion; second, fingertip skin-slip direction combinations that are either
in the same direction or the opposite direction as represented by the four VR objects and movements;
and third, four haptic mappings that were different in 90 degree intervals with a no-haptic
baseline. For the passive hand movement, the virtual object translated or rotated on its own,
while in the active hand movement, participants moved or rotated their hands
in the specified axes. Interestingly, when the hand was moved actively to perceive the object,
there appeared to be no difference in how the haptic mappings were perceived aside from the no-haptic
baseline, despite that skin-slip directions far exceeded the average JND found in Study 1.
Also, in terms of proprioception, when the haptic mapping was congruent with the visuals,
participants appeared to perceive higher congruency in the passive hand movement than in the active
hand movement. We conducted a third study to see how width-change would be perceived together with
skin-slip, comparing dynamic width-change to a static width baseline. Participant ratings on
realism, immersion, and enjoyment indicated that no difference was perceived between
the two conditions. In the follow-up interview, surprisingly, we found that all but one out of the
12 participants perceived a width change in both the static and the dynamic width change conditions.
We interpret these findings as an indication of cross-modality sensory illusion as described in
literature. If you are interested in more details on each of the three studies, please refer to
our paper. We developed three demo applications demonstrating interactions enabled by SpinOcchio.
The Potter's Wheel application uses grip force sensing to allow users to
manipulate the shape of the clay through squeezing, while simultaneously feeling the
speed and the width grow and shrink as they feel the contours. In the Weight and Pulley
application, the user can pull the rope on a pulley holding a weight on the other end.
By squeezing the rope, participants can pull the weight off the ground, and by forming a loose grip,
participants can feel the rope slide through their fingers as the weight falls back to the ground.
In the Material Laboratory application, participants can feel an illusion of material
with different elastic properties. By pulling different material samples, participants can
feel rigid and stretching material slip through their fingertips at varying rates and thickness,
proportional to their hand movements. Now here are some of the key takeaways from our work.
First, in terms of haptic perception, the sole number of sensory input channels does not
guarantee better haptic discrimination. We saw from our results that generally, participants needed
a larger angle change to perceive a difference when involving two fingertips over a single one.
Therefore, other sensory cue should be considered when rendering a desired effect for multi-channel
haptic interfaces. Second, in the presence of multiple sensory feedback modalities, a known
perception may be masked or influenced by another strong feedback modality.
This means that as seen in our second study, because participants could not
well-differentiate between different haptic mappings that are pointing in very different
directions, future work on skin-slip perception in multi-sensory environments may focus not on
investigating replicating skin-slip directions, but on other aspects, such as skin-slip texture.
Finally, in a multi-sensory feedback environment, unexpected haptic illusion feedback may be
perceived due to a strong feedback modality, such as visuals. This means that as shown in our third
study, because participants perceived grip-width change even when no width change was involved,
future work on width change and skin-slip haptics in VR may focus on not investigating actual width
change perception, but on the haptic illusion of width change and how to maximize its effect.
And with that, I hope you found some parts of our work interesting
and thank you for taking your time to listen. I'll be happy to answer your questions.