Hello, my name is Vivian Shen
and I'll be presenting Mouth Haptics in VR Using a Headset Ultrasound Phased Array
This is work done with Craig Shultz and Chris Harrison
as part of the Future Interfaces Group.
Rendering haptics to the body is an open challenge,
the mouth is of particular interest to us
as it is second in terms of tactile sensitivity to the fingertips.
We wanted to take advantage of this unique opportunity
to add fine-grained haptics effects
to a head mounted virtual reality device.
Rendering haptics on the mouth has also been difficult to accomplish
in a practical and consumer friendly manner.
Here is a video of four systems from prior research.
that utilize a wide variety of sensors and motors
to create visuo-haptic VR mouth experiences
In this research, we iteratively developed
a new way to achieve haptics on the mouth,
using a thin, compact, beamforming array of ultrasonic transducers
to focus acoustic energy onto the mouth.
Here we have depicted our first prototype,
our proof of concept work that was used in our studies,
and our future work.
We also developed a software simulator in Unity
to be able to manually focus the beam in space.
Here, the red bar represents where
the average person's lips would be relative to the transducer array,
depicted as the 64 colored spheres.
The black sphere represents the focus node,
and as we move it through space using the arrow keys,
the transducer spheres change color according to the phase offset that they are in.
We explored several effects, including single impulse
impulse trains,
swipes on the X, Y, and Z axis,
and persistent vibrations.
Our haptic sensations can be felt on the lips,
the teeth, and the tongue.
Here's a table depicting the six animation types that we defined,
seen here being animated on the lips.
We conducted a pyscho-physiological perception study
that consisted of modulating the amplitude of a single impulse animation
and asking if the user could still feel the node.
We ran 8 users through this study.
To find the threshold of detection,
we utilized the double staircase method as illustrated here.
This is an example of the staircases
pulled from one of the participants of our study.
Here we have depicted the varying thresholds
measured from the perception study.
The staircases were repeated at different modulation frequencies
on the lips, teeth, and tongue.
At the lips and teeth, the detection threshold
was lowest at 50 Hz,
meaning that our ultrasound phased array
had the strongest effect at 50 Hz at those two locations.
On the tongue, the effect was strongest at 100 Hz.
We also conducted a spatial acuity study,
where we played two spaced apart 500 ms impulses
also at the lips, teeth, and tongue.
We found that there was essentially
zero spatial acuity on the teeth and tongue,
as there were zero converged staircases.
On the lips, the staircases converged around 9 mm,
meaning that users could distinguish nodes that were at least 9 mm apart.
We also conducted a user experience study
by running users through 3 different scenarios
with 12 interactions using our 6 animations.
We played each interaction twice for the user,
once with mouth haptics and once without,
and we ran 8 users through this study.
I'll be going through every interaction now.
First, we had the user starting off in a haunted forest scenario.
As the user walked through the spiderwebs,
they feel an X swipe animated on their lips,
as if the web was being dragged off their face.
In a small clearing,
a spider jumps on their mouth,
which manifests as random X and Y swipes to the left of the mouth.
This simulates the spider scurrying around on the user's face.
The user then uses a flare gun to shoot a spider on a rock.
When the spider explodes into goo,
the user feels random impulses on the lips.
Finally, the user is forced to walk through dripping spider venom,
which manifests as Y swipes on the lips.
The next scenario, the user is placed in a school.
As they lean down to take a sip from the water fountain,
they feel rapid random impulses simulating the water hitting their lips.
In the school break room,
the user sips from a mug, and feels the liquid entering
their mouth as Z swipe animations.
They also take a smoke from a cigarette,
which is animated as a persistent vibration node
wherever the cigarette meets their lips.
Finally, they brush their teeth after the break.
As the user uses the toothbrush,
they feel back and forth X swipe animations on their teeth.
In the final scenario,
the user races down the road in a motorcycle.
They feel wind being animated on their lips
as two X swipes moving in opposite directions.
The user comes across some boxes on the track.
As they run into and break these boxes,
they feel a piece of the box hitting their lips,
as a short persistent vibration.
It begins to rain, which manifests as slow, random impulses
on the user's lips, as if the raindrops were hitting their face.
In the final interaction, the rain has left puddles on the track.
When the user runs through these puddles, they feel
rapid random impulses on their lips, to simulate
the splash of the puddle hitting their face.
After each interaction was played twice, we had the user answer
a Likert scale survey about the experience.
Here are the results of the inter-trial survey.
As depicted in the bar graph, with mouth feedback dominated
every category of realism, localization, tactility, reaction, immersion,
and visuo-haptic match, for every interaction.
We also had the user fill out two more Likert scale surveys
after they finished the study.
In the first of these surveys,
we asked them to rate the realism, immersion, and fun
of the no mouth and with mouth feedback interactions,
and then asked them to rate their preference.
With mouth feedback won in every category,
and zero users preferred no mouth feedback.
In the second post-study survey,
we utilized questions regarding the 22 dimensions of haptic experiences,
to measure the autotelic, expressivity, immersion, realism, and harmony
of the visuo-haptic effects.
Here are the scores across all categories
for the ultrasound haptic feedback device.
Thank you for coming to my presentation.
I will now take questions.