Hello, everyone, I'm Yannick, and I'm happy to present our
paper on using pseudo stiffness to enrich the haptic experience
in virtual reality. On behalf of my co authors, Steeven Villa,
Albrecht Schmidt, Sven Mayer, and Florian Müller. In this
work, we present and investigate a novel technique for changing
the perceived stiffness of objects based on a visuo-haptic
illusion elicited through virtual reality. This work was
inspired by the fact that in our everyday lives, we often rely on
our perception of stiffness when assessing our environments and
making decisions such as checking the ripeness of a fruit
or choosing a suitable mattress to buy. And while these haptic
sensations are natural aspect of how we experience the physical
world, virtual reality is generally dominated by visual
and auditory stimuli only thus in turn losing a fundamental
part of the experience. While active devices can help bring
haptic feedback to virtual environments, they're usually
not able to render stiffnesses accurately since these require
extremely fast changes of force outputs. The ones that do are
usually very limited in their workspace or very bulky and
complex, thus are mostly used in very specific application cases.
passive devices like passive props, which you can simply
distribute around the physical space of our VR scene allow for
very accurate representations of object stiffnesses, but of
course, they cannot adjust their properties dynamically. So they
are very limited to their predefined objects and use
cases. This is where haptic illusions come in. They allow us
to more easily adjust some aspects of the haptic experience
by manipulating our perception in most cases through visual
cues. Illusions have already been shown to work to adjust the
perceived shape, size, weight, texture, and even stiffness of
objects. However, the approaches to alter stiffness so far mostly
relied on visual deformation of the surface, which in turn,
impacts the visual experience and restricts the flexibility
and applicability of the approach. We present a novel
method that can alter the perceived stiffness of objects
based on a visuo-haptic illusion that does not require bulky
hardware, and, more importantly, actually remains undetected by
the user, which allows it to be easily integrated into a broad
range of systems without negatively impacting the
remaining experience. Our proposed technique modulates the
perceived stiffness of objects by dynamically adjusting the
control to display ratio of users hand movements while
interacting with objects in VR. Control to display ratio or C/D
ratio describes the relationship between actual inputs which in
this case will be the actual movement of the hands, and the
resulting change in the display which here is the movement of
the virtual hand representation. Based on the visual dominance
over proprioception, people's sense of where their actual
hands are in space will drift towards the visual
representation. This effect has already been thoroughly explored
and utilized in research to redirect users of VR systems to
a physical prop while they're trying to grasp a virtual object
which is not directly located in the same spot. In our approach,
we modulate the movement gains of the hand in accordance with
the desired change in stiffness while users are pressing down
onto an object. So if a softer object is desired we increase
the movement gains of the virtual hand during the press,
resulting in the virtual hand pressing further down onto the
object. And for a stiffer object, we reduce these gains,
restricting the movement of the hand partially while the user is
trying to press further. To evaluate this approach, we
conducted two controlled studies. First, we needed to
find out how far we could actually adjust the C/D ratio of
real to virtual hand movements without the user detecting the
manipulation. And second, we wanted to find out how the C/D
ratio adjustments then affect the stiffness perception of
users when they press on a compliant object in VR. We used
an HTC Vive pro with an attached Leap Motion Controller to
display a VR scene and track the user's hands. To render the
haptic feedback necessary for our controlled studies, we
relied on a Novint Falcon, a haptic device that is able to
adjust its forces fast and precisely enough to render
stiffnesses accurately albeit in a very small workspace. For the
first study, in which we aim to find out how much we can
actually increase and decrease the movement gains without
detection by users, we use the method of constant stimuli. For
this we presented different fixed values of C/D ratio
manipulations, which we determined by a pilot study and
asked participants whether the movement of the virtual hand and
the real hand matched up. We varied both the physical
stiffness that the haptic device was rendering, and the C/D
ratios and repeated each of these combinations a total of 10
times, we split decreased and increased C/D ratios into two
separate blocks, which we counterbalanced and randomized
all trials in each block. We then calculated the ratio of
detection of the illusion for each combination and fitted the
results to the psychometric functions you can see here. The
detection thresholds are defined as the C/D ratios which would be
detected 50% of the time, we calculated the resulting mean
thresholds which are approximately 0.7 and 3.5 times
the actual movement of the hand. We used these boundaries to
inform the maximum values used for C/D ratio manipulations in
the second study, in which we aimed to find out how much we
are able to change the perceived stiffness of objects using this
illusion. We again use the method of constant stimuli in a
two interval forced choice task Each trial consisted of two
presses on the haptic device, one of which was the reference
stiffness rendered by the haptic device without any additional
manipulations. For the other press, both the C/D ratio as
well as the actual physical stiffness were varied.
Participants were then asked to determine which of the two
virtual objects felt stiffer. Each combination of reference
stiffness, physical stiffness manipulation, and C/D ratio was
repeated five times resulting in 525 randomized trials per
participant. We then fitted the results averaged across
participants to psychometric functions, and determined the
amount of actual stiffness which could be offset by our illusion.
We observed the largest increase of perceived stiffness in our
softest condition, in which the illusion resulted in the object
feeling roughly 9% stiffer, the largest decrease in perceived
stiffness occurred in the hard condition in which the object
was perceived to be 28% softer. With these large effects on
perceived stiffness of objects, while remaining undetected by
users, we argue that this illusion can be utilized in a
wide range of applications and can be applied to various haptic
feedback devices to enrich the haptic experience in VR. For
example, the illusion can enhance the variability of
custom built passive devices, such as springloaded buttons. Or
we can leverage the illusion in combination with common everyday
items such as pillows, or cushions, to stand in for
variety of virtual objects. And finally, the visuo-haptic
illusion can enhance active haptic devices, such as
Encounter-type interfaces, which would lack the refresh rate to
render realistic stiffnesses on their own. In conclusion, we
found that out presented visuo-haptic illusion is very
effective at altering the perceived stiffness of objects,
while still remaining undetected by the user and thus not
negatively impacting the remainder of the VR experience.
Therefore, our work contributes to the body of work on haptic
stiffness and can be used as an easy and scalable option to
extend the range of applications of active and passive haptic
feedback systems. If you have any questions or comments,
please refer to our paper or feel free to reach out to me via
email.