Hi, my name is Shan-Yuan Teng. I'm going to talk about our work touch&fold,
a foldable haptic actuator for rendering touch in mixed reality, this is in collaboration
with my colleagues Pengyu, Romain, Joshua and my advisor Pedro Lopes at the University of Chicago.
we engineered a foldable nail-mounted haptic device for mixed reality.
Here the user follows a guide to repair their bicycle,
this is the virtual interface they see, when they touch this button
our haptic device unfolds a cover that pushes against the user's fingerpad, then
it tucks away, leaving the fingerpad free
the guide then instructs the user to use a screwdriver to unscrew the brake pad
Because our device folds away from the fingerpad,
It allows the user to manipulate tools without impairing haptic sensations
why did we build this?
mixed reality allows overlaying digital contents with the real world
however, in terms of haptic feedback, it's quite different from virtual reality
because there are not only virtual interfaces
but also existing physical objects throughout the interaction.
As we see many devices render haptic feedback for VR such as using vibration,
motor & pulley, or braking. Because they cover most of the hand
These decrease dexterity and blocks haptic sensation from the real world,
and they are not suitable for MR.
To minimize the interference with the real world,
film-based haptic interfaces leverage electro tactile and fluidics
however, this can still impact our haptic sensation because our fingerpad is so sensitive.
So here we ask: is there a way to keep user's fingerpad free?
Our approach is to touch
and then fold
when the user touches a virtual bicycle tire wearing our device
our device pushes against the fingerpad to render the sensation of contact.
When the finger leaves the virtual tire, our device folds back onto the nail
so that the user can touch and feel the texture of the real tire.
Let me show you how we engineered this, the device consists of
3D printed shell,rack & pinion, Our custom PCB, bluetooth antenna, And a battery
the cover slides and rotates through the hinge,
and brings down a vibration actuator to touch the fingerpad.
let me explain the mechanism in a slow motion video
first, the pinion unfolds the cover when the cover passes this hinge, it folds inward
finally it is pushed flush against fingerpad The
pinion rewinds the cover when the interaction is finished
Our device renders the sense of contact for a virtual surface
The cover unfolds and apply pressure onto the fingerpad
Our device renders variable forces , for example,
a button feels different pressure when pressing down,
the cover stops when touch is sensed by the force sensor, creating a light force,
then the motor applies more pressure
Our device can render textures as well. For rough textures, like a corrugated cardboard,
Our device rocks the cover back and forth on the fingerpad
For fine textures, like a piece of sand paper,
Our device brings down the vibration motor that vibrates on the fingerpad
we conducted a user study comparing with a prior work which is fingernail vibration
that also leaves fingerpad free. The participants
felt our foldable actuator provided more realistic haptic experiences.
In another study we found our device preserved the dexterity and haptic perception
in an MR experience for repairing a bike brake.
We demonstrate feeling feedback from virtual piano keys, and from keys on a physical keyboard.
By wearing two of our devices, when the user pinches a slider, both of the devices unfold
and touch fingerpads, creating a sensation of touching something in between finger and thumb.
This is touch& fold, the system that highlights the importance of designing haptics with
fingerpads free to feel the real world. Lots of prior haptic devices only work in virtual reality,
we were excited to explore solutions that work in the virtual and the real world as well,
so we don’t have to choose sides and can enjoy haptics in whichever world we are.
Thank you very much