To me, what's been happening in research on vibrotactile rendering in the last years is really exciting.
So, using vibration,
we're able to render realistic surface textures where there are none.
We're able to provide a button with arbitrary affordances
by creating virtual force pressure curves. We can even render textures
in mid air or create hyper objects which have virtual tactile gestural affordances.
We can make rigid displays appear compliant, including the compliance
of the ground we walk on to create virtual grounds in virtual and augmented realities.
We can make hard objects appear soft.
We can create a virtual experience of bending and torsion
and generally design the material properties of the objects and devices we interact with.
It's so exciting, really, really exciting.
How is this possible? Well, material experiences are created through interaction. For example,
when I lift a cup, I apply pressure to the cup based on the tactile feedback I receive. I'll
modify my grip, again leading to new tactile experiences. This closed-loop behavior enables
material experiences to emerge.
So let's turn on the audio for some of these videos I just showed.
In these videos, the vibration is presented as audio.
You'll notice that there is similarity between how these systems sound,
even though they're created by different research groups at different times.
The reason for this is that all these systems provide vibration with a common structure.
Vibrotactile impulses are synchronized with user actions. Specifically, when the user acts,
this action is sampled. Signals are generated accordingly. Based on this type of feedback we
receive, we will modify our actions which again is sampled and leads to new vibrotactile feedback.
This augmented closed loop enables virtual material experiences to emerge.
Let me explain this with a concrete example. Let's say we wish to add friction to a slider.
We would first decide upon fixed points at which to trigger pulses.
Then, we sample the movement of the slider in real-time with an analog sensor.
Whenever one of these trigger points is crossed, we play back a vibrotactile pulse.
this way, the motion of the slider is transformed, in real time,
into a vibrotactile signal, which is experienced as friction.
These systems are non-trivial to implement as they require both high analog resolution on the output signal
as well as high temporal resolution with regards to sampling human action and generating
corresponding vibrotactile feedback. To make such a system more accessible to the general public, and
because we need this type of device for our own research, we created what we call haptic servos.
Haptic servos are self-contained embedded platforms that create vibrotactile signals
based on human actions. So, using one of our haptic servos, any of the systems I showed
at the beginning, any of those videos, they can be implemented in a matter of minutes.
The haptic servo samples human actions using an analog sensor; this can be considered the
servo's control signal. A microcontroller generates corresponding output signals,
which are amplified to drive a haptic actuator. The human then closes the loop. We minimize system
latency by removing all non-timing critical elements. For example, changing the parameters of the haptic signal is done asynchronously.
A fun feature we implemented is that haptic servos
respond to pulse control like a regular servo. You can find more information on that in our paper.
Haptic servos are open source. You will find the link to the GitHub repo at the
end of my presentation. They are prototype devices, but we have successfully used them
in a bunch of projects. One of them will be presented by Nihar Sabnis here at CHI,
and the other was presented at Augmented Humans. And all the others are still kind of ongoing.
However, as I'm saying, they're prototype devices, so they come with a bunch of limitations,
and they're not perfect in any way. And we hope to address many of these in the future.
And they're open source, so we invite anyone interested to collaborate with us in
making them better. Or if you want your own version, they are open source. You can fork
our repo and replicate it, modify it, and create your own devices based on our work.
So, in addition to this hands-on open source contribution, this project also points towards
other interesting facts about perception. One of them is it appears that a broad
range of vibrotactile effects actually rely on the same underlying perceptual mechanism.
Vibration proportional to changes in human action seems to be the way we expect tactile
information to be organized so that we can make sense of them as a material property.
It's also interesting to note that the final experience that we have
is driven by the human action, which is augmented. For example, changing input from
isometric to isotonic has a much larger effect on the resulting experience than manipulating
any parameter of the vibrotactile feedback. We discussed this in much more detail in our paper.
To us, haptic servos are the beginning of creating an infrastructure which allows us to
experiment and design with material experiences. We're super happy that we're not only presenting
the output of our research, but also the tools which we created for ourselves to
conduct research. And it's our sincere intention and hope that others will benefit from this.
So, I'm looking forward to chatting with many of you in person at CHI.
This is work by the Sensory-Motor Interaction group at the Max Planck Institute for Informatics.
My name is Paul Strohmeier. My fellow research group members Nihar Sabnis, Dennis Wittchen,
and Courtney Reed worked on this with me. We also had support from our neighbors of the
HCI lab of Saarland University, in particular from Jürgen Steimle and Narges Pourjafarian.
Thank you for your attention. See you at CHI