Hi, my name is Jasmine Lu and I’m presenting our work Chemical Haptics, a new approach to generating haptic sensations by applying chemicals to the skin. This is in collaboration with my colleagues Ziwei Liu, Jas Brooks, and my advisor Pedro Lopes at the University of Chicago.
Here, a user is in VR walking through a corridor when an explosion occurs! This makes their arm interface malfunction with electrical sparks flying out of it.
Our wearable device creates the sensation of electrical sparks tingling your arm by delivering chemicals directly to the users skin. Specifically, we use the chemical SANSHOOL to create a TINGLING sensation. Our chemical approach allows for a wide variety of other sensations too such as cooling, warming, numbing, and stinging.
Now, let me take a step back to explain how we arrived at this idea.
Haptics explores how to expand touch sensations via devices.
This is primarily accomplished in haptic devices by stimulating the mechanoreceptors in our skin. Often, devices use vibrotactile methods or force-feedback to accomplish this.
However, our skin sensations arise from more than just mechanoreceptors. We also have thermoreceptors for temperature, nociceptors for pain, and several others. These receptors have not been explored as much for haptics.And most of these explorations have been done only with heating elements or electrotactile displays.
Our work explores another approach to target these receptors and thus, enables another way to achieve haptic sensations.
Specifically, we stimulate these receptors with CHEMICALS.
I’ll return to the walkthrough video and play the complete virtual reality experience to explain.
As you saw, there’s a blue liquid circulated on the arm...The skin is absorbing the chemical ingredients of the liquid via channels that are open to the skin. As I explained earlier, this creates a tingling sensation.
Here we use the chemical Sanshool which you may be more familiar with in the form of Sichuan peppercorns. Sanshool causes a sensation of tingling by stimulating receptors that respond to light touch.
The reason why this works is that… Just as you can trigger receptors physically such as with light touch, you can trigger those same tactile receptors CHEMICALLY.
Now let me continue playing our video walkthrough and you can see four other sensations we found with this approach.
As the VR experience continues, the user walks outside into a wintery mountainous landscape.
Here we use the chemical Menthol found in mint. Menthol causes a sensation of cooling by stimulating receptors that respond to low temperatures. Again, rather than stimulate the thermoreceptor with cold, we stimulate it chemically.
As the user turns to their right, they see the entryway to the nuclear reactor which is on the brink of meltdown. They use their arm interface to open the doors and but then, hear a glitching sound – their arm interface is malfunctioning and renders a phantom arm
Here we use the chemical Lidocaine which is commonly used in medical applications. Lidocaine is an anesthetic that numbs by inhibiting sensory nerves. This reduces haptic sensation.
With a malfunctioning arm interface, the user must find an alternative way to access the room and shut down the nuclear reactor. Luckily, they find a manual lever on their right which they pull to open the door. Now, they walk into this hot and steaming reactor room.
Here we use the chemical Capsaicin which is found in chili peppers. Capsaicin creates a sensation of heat by triggering thermoreceptors that respond to hot temperatures.
We featured those four chemicals in our walkthrough, but we also found 1 other very compelling chemical induced sensation.
In this VR experience, a user pours a liquid on an open wound. Here we use cinnamaldehyde to render a sensation of stinging. Cinnamaldehyde triggers several receptors – noxious cold receptors, itch receptors, and pain receptors.
To review, we found that we could target a variety of thermoreceptors, pain receptors, and mechanoreceptors with chemicals and even found we could inhibit sensory nerves too.
Now that we’ve shown the sensations we achieve with chemical haptics, I’ll show you how we delivered the chemicals to the skin
Here, we show a material stack of the device system. The top layer is silicone. And the bottom layer is the skin. In between, you see two types of channels - green and blue.
The green channel moves the liquid away from the pump and into the blue channels. These blue channels are different because they are open to the skin. This allows for direct contact with the skin, so the skin can absorb the chemical.
This is when the haptic sensation appears – as the chemical moves across these open channels.
Now, our system pulls the liquid through the remainder of the open channel, back into the green channel and again, into its initial reservoir in a closed loop. We use this closed loop design to maximize our chemical usage and enable longer stimulation time.
We implemented this design in a self-contained sleeve device. We have the liquid channels on one side and electronics on the other.
Here, you can see how all the electronic components are embedded in the silicone. we include the micropump, pump driver, and microcontroller - all the components needed to drive our device. In addition, we also use thermistors as liquid sensors to determine when liquid is in the open channel.
As you’ve seen in the walkthrough, we’ve also made a headset version that targets the cheeks.
As the first exploration into the territory of chemical haptics, we were really excited to look at how users feel when stimulated with our haptics approach.
In our first study, we focused on understanding what sensations arise and when they arise while chemicals are applied to the skin in isolation of any other stimulus or interactive context.
Using this data, we were able to narrow down a set of chemicals to use in interactive applications.
Which is what we validated in study 2.
We asked participants to experience VR scenes with and without chemical haptics and rate how immersive they felt.
Consistently, participants rated the experiences as more immersive with chemical haptics than without. And all participants stated that they preferred having chemical haptics over no haptics.
To conclude: Today, I’ve presented a new approach for generating haptic sensations
It relies not on physical actuators that transmit vibrations, electrical impulses, or heat to your skin; but by delivering chemicals that trick your receptors into generating sensations that your body interprets as haptics.
In many cases, what’s interesting is that these offer a new way to achieve sensation. Sometimes with much less power than say a heating element.
Also, rather than combining different devices together, we can achieve multiple sensations with the same hardware solely by switching the chemical.
Now we think there are many ways we could improve current expressivity of chemical haptics.
One would be mixing these compounds to see if new sensations arise. Another one would be to explore custom engineered chemicals for haptics.
The chemicals you’ve seen today can easily be bought in stores – this has the benefit of being skin safe but has the downside that they were not meant for haptics.
Finally, I want to tell you why we’ve been exploring an idea like chemical haptics.
Our lab’s mission is to integrate interactive devices with the human body,
On the right side, you can see an example of our work on miniaturizing force feedback sensations…
instead of adding more electronics to the body that externally induce sensations, such as pushing someone’s hand with a exoskeleton motor, we...
add only the minimum components to trick the user’s biology into internally inducing the sensation this is precisely what we have been exploring with our line of work on chemical haptics.
Last year, at CHI, my coauthor Jas Brooks presented our first foray into chemical induced haptics, they generated hot and cold sensations in VR as user’s breathed in chemicals.
With chemical haptics, we go one step further and explore both a wider range of sensations beyond just hot and cold, and explore chemically induced sensations on the user’s skin, the body’s largest haptic organ.
Ultimately, we see this as the FIRST STEP for chemical haptics. We’re excited by the potential to broaden the ways we approach haptics, particularly in ways that create novel sensations beyond just vibration and pressure that might be more deeply connected with our underlying biological mechanisms.
Thank you for listening to our presentation! We encourage you to further engage with our work at our UIST demo session, by reading our paper or by remixing our device, which is entirely open source.