(coughing)
- Hello, my name's Imrak Choi from Stanford University
and this work was done with wonderful collaborators,
Heather Culbertson, Mark Miller, Sean Former
from Stanford University and Alex Olwar from Google.
Thanks to all of you and let me start the presentation,
Grabity, a wearable haptic interface from simulating
weight and grasping in virtual reality.
So we are interested in haptic technologies.
For the last two decades, haptic technologies
were focused on surgical robotics and teleoperations.
Researchers focused on high performances
in terms of precision position tracking,
force rendering and fast control loop
for specific manipulation task.
The haptic device is used to be desktop mounted
for creating forces from outside.
Because these devices are grounded mechanically
to the outside, often times people call them
grounded haptic devices.
However there has been much excitement about
commercial VR and how haptics apply to it.
If you look at this video of Oculus, we can see that
there are many objects that users may
want to interact with naturally.
We and many others believe haptics is key enabling,
to enabling this kind of rich physical
interactions with virtual objects.
So for general interaction in commercial VR,
wide motion range, mobility and low cost
are perhaps more important properties.
And because of that there are many research projects
recently about ungrounded haptics.
So we see that the form factor of
haptic devices needs to change.
We believe that ungrounded haptic devices
are more suitable for commercial VR haptics
because they are wide range of,
they have wide range of motions.
However there is a limitation among
ungrounded haptic devices.
Ungrounded haptic devices cannot provide external forces
because they are not grounded to outside
through mechanical linkages.
And we think this is a big limitation for VR applications.
Let me give an example, when a person grasps an object,
there are grasping forces to the thumb and fingers
and also weight from the mass of the object due to gravity.
There have been many haptic devices developed for
haptic force feedback for research or commercial purposes.
However these existing haptic gloves provide
grasping forces only and do not render object weight.
This is not desirable because first, it is not realistic
and users don't know when they lift the object
so it is bad for manipulation.
An ideal haptic device for commercial VR
would be an ungrounded haptic device with the capability
of rendering external kinesthetic forces.
And our goal is this paper is that building
an ungrounded haptic device for grasping
with the sensation of weight.
First we need to think about how we perceive weight
basically it is a combination of multiple
sensory systems on the human body.
There are many sensory receptors for weight perception,
first, every muscle has muscle spindles inside
and they sense force by measuring pressures.
Second, Golgi tendon organs are stretched
and feel the tension.
Finally there are mechanoreceptors in skin
and they sense sheer forces and pressures.
So by sensing various pressures and tensions
in different places on the body,
at the same time we can perceive weight.
And our goal is simulating weight feedback by just using
the sensation from mechanoreceptors on fingertip skin.
Recently there is an interesting research topic
about this called asymmetric vibration.
Researchers found that human senses virtual
kinesthetic force through asymmetric vibration.
Arnemiya et al. Investigated this virtual force effect first
in 2008 using their slider crank linkage device.
More recently Rekimoto found the same effect
using a voice coil actuator which is much
smaller and lighter than Arnemiya's device.
Then Culbertson et al. Have modeled the asymmetric vibration
from a voice coil actuator.
So far there are many, there are not many
applications using this illusion and
we thought it can be a good trial to use
this illusion for a haptic device.
First, let's look at grasping an everyday object
to see the haptic feedback we need to generate.
We can see there's a contact, grasping,
gravity, force and inertia.
And this is our device Grabity.
So there are four types of haptic feedback
that Grabity can render.
It renders essential haptic feedback during on.
So first when users make the contact to the virtual object,
the same voice coil actuators used by Culbertson et al.
To generated asymmetric skin stretch are used
here to generate transient vibration.
Once the user has grabbed the virtual object.
The break locks the finger motion to create grasping force.
When the users lift the object, the voice coil actuator
make asymmetric vibration to simulate weight sensation.
If the users shake the virtual object,
the magnitude of asymmetric vibration changes
based on the acceleration of the hand.
And now lets look at how the device works.
There are three main bodies on the device,
there is a sliding part, base and swinging part.
The index finger and middle finger are mounted on the
sliding part, and the thumb is mounted on the base.
There are two types of actuators, there is a
unidirectional break next to the index finger
and there are two voice coil actuators on the swinging part.
And the distance between the index finger and thumb
is measured with an encoder and there are
retroreflective markers for global position tracking.
So the device weight is about 65 grams
and the pinching distance is between 30 and 100 millimeters.
The voice call actuators are connected to bearings
so they rotate passively to align their
directions normal to the ground.
Because of this, we can render the gravity force
even while rotating the device.
However the device doesn't render the gravity forced
to the normal direction on the finger pads
because the asymmetric vibration illusion
works effectively only with the lateral skin stretch.
To generate grasping force feedback,
we brought the brake mechanism from Wolverine.
It is a unidirectional brake mechanism
create very rigid stiffness.
We selected this brake mechanism because it is very
simple to control and lightweight
while it also creates large grasping forces.
Two voice coil actuators give various haptic feedback
such as touching, gravity and inertia.
Voice coil actuators are basically speakers
and we can generate vibration with various frequencies
and amplitude independently using the actuators.
The impulse current signal show
how to create virtual forces forces for weight simulation.
We see that the current goes negative
with our step function and goes positive linearly.
This pattern makes the voice coil actuators
move downward quickly and moves up slowly.
Then the asymmetric vibration generate
the downward virtual force.
The virtual force perception was dependent
on how we grabbed the device.
The finger pads should be placed on
the mount part correctly for skin stretch.
So we attach a sand paper on the finger mount
surfaces for better friction.
We connected gravity to our VR system,
we used Oculus Rift for this play and OptiTrack for tracking
and we used CHAI 3D Haptic Engine
for graphic and haptic rendering.
We render some simple 3D shapes with
different sizes and different weight.
So after implementing the device, we had two main questions.
First we wanted to measure how strong
the perceived virtual force is.
So and we had to know this to render haptic feedback
using the device and we also wanted to know
if it is controllable.
And second, we wanted to know if the device
really worked for weight simulation in VR.
To answer the first question we measured the
asymmetric vibration first, we measured accelerations
using an accelerometer mounted to the device
to see the asymmetric vibrations.
The top figures show the impulse signal
and the bottom figures show the acceleration output
with different current amplitude.
The five figures in the top low, row,
we see that the magnitude of negative acceleration
keeps increasing while the positive
acceleration looks the same.
From the five figures below we see that the negative
acceleration doesn't change anymore
and the magnitude of positive acceleration keeps increasing.
To see the effect of the current magnitude
we made another cloud with the same data.
The X axis is the magnitude of the current
and the Y axis is the acceleration assymetry
which is the difference between the absolute values
of the maximum and minimum acceleration.
And we see that the assymetry only increases
within the range between point two and point four.
So we decided to use the three values in this range
for virtual force rendering.
12 people participated for the virtual force measurement.
We used staircase method to calculate
the perceived virtual forces.
Users compared the weight from the
virtual force with a real mass.
For better explanation we give,
if we give downward asymmetric signal
to the voice coil actuators, users will receive
vibration but also the downward virtual force.
To measure the downward virtual force,
the symmetric signal on the right side
gives the reference haptic feedback
which is just vibration but same frequency
and magnitude with the left signal.
The participant will say the left one is heavier.
Then we add a rear mass to the right signal
and users can compare the weight until
they cannot tell the difference.
We measured three types of virtual forces.
We measured downward forces, upward forces
and the force with difference between upward and downward.
We measured with three different magnitude of current
and so there were total nine conditions.
And this is the result, we found that the device
can generate downward force up to 15 grams
and upward force up to eight grams.
The device also can generate various amount
of virtual forces by controlling the magnitude of current.
However the result have huge variant depending on users.
After the first task, we wanted to get
qualitative feedback about how the device works
when grasping object in VR.
To do this we gave users a simple resourcing task.
We rendered three blocks with different weight.
Users picked up and felt each block's weight and they
placed the blocks on top of the shelf in order of weight.
For weight rendering of three blocks we used
the same signals in the first task.
From the first task, the three perceived
downward forces were about six grams, 12 grams and 15 grams.
There were five participant about this
and they did six trials for each.
After the weight sorting task, they used the device
for three minutes for qualitative feedback.
This computer metric shows the result.
Users responded correctly with 60 percent of chance.
There were, there was some confusion.
Especially users were confused
between medium and heavy weight.
This gave a similar trend between the task one result.
The medium and heavy weight were too close so it
was almost the border of the just noticeable difference.
The following qualitive feedback tell how users thought.
First, vibration was apparently interferes
some users experience.
Some users also felt that it was really difficult
to differentiate the three weights.
Makes sense because the amount of virtual
forces were pretty small and there were also
vibrations interfering the perception.
Users, users likes the fact that
gravity was not grounded to outside and
they had large range of motion.
Finally they were surprised about the
haptic illusion in general.
So there were some limitations of the device.
First the vibration bothers some users,
however users need to feel the vibration
first to feel the virtual force.
Second, the virtual force strengths are hardware-dependent
and currently limited to negative 15 to eight grams.
Also the haptic illusion was varied a lot,
depending on how users grabbed the device.
And finally this mechanism works with asymmetric
skin displacement so it worked well for the later,
lateral directions of the finger pads.
There are two topics we are interested in the future.
First, we still don't know the exact mechanism
how we sense the virtual force.
But at least we know that the amount
of virtual force is quite small.
So we want to try to find how we can maximize
the virtual force by changing actuators
or the current signal patterns.
Also we would like to do more testing with users
to see the trade off between different comfort
and the virtual force amount.
In this talk we introduced Grabity
and Grabity renders four types of haptic feedback
such as contact, grasping, weight and inertia.
Grabity renders control over virtual forces.
Users feel various weight from the device
but vibration currently limits the,
some realism.
Thank you very much for your attention
and are there questions?
(clapping)
- [Tak] I'm Tak Kijan from Kyung University,
my concern is that is it really realistic?
Because I think, it can generate direction of force
but
I'm not really sure that it really generate
the feeling of, feeling of, of gravity.
So it's a little bit different from actually
generating directional force I guess.
So realism, is it really realistic?
As if it, you, someone really feeling
as if they are grabbing something?
- So your question is about the realism
about the force for gravity rendering?
- [Tak] Yes, I think there is a difference between
just showing directional force
and giving realistic feeling of grabbing.
- So mostly people said it was almost like a pulling down
or a pulling upward force like continuously
and then we just used that effect for the gravity rendering.
But people said, even we, so from the first task
we measured vibration, the virtual force with the real mass
but some people said the virtual force is more like,
is more than real mass, like even we,
like put some weight, they can still feel
the forces upwards for example
so there was no specific mechanism I understood from there.
But
we used that effects from gravity rendering.
And then people, obviously people felt some,
like amount of weight, like as 15 grams.
So okay.
- [Quan] Hello, Quan Sannity from EPFL Switzerland,
thank you for your talk, wondering about
the frequency used for the vibration
and if there is any effect increasing
or decreasing on that frequency?
- So, so we feel the vibration
with the range of like a zero to like 100 hertz
but this, for this skin stretch, asymmetric skin stretch,
I think 50 hertz was kind of the maximum
we can tell the direction and some people
said 30 hertz was good enough but mostly
about 40 hertz was the maximum like force generated.
- [Man] Thanks, let's thank the speaker again.
(clapping)