Hey everyone! My name is Jas Brooks, and today I'll be talking about how to
create temperature sensations using a special nerve in our body called
the trigeminal nerve. This work was done in collaboration with
my colleague Steven Nagels and my advisor Pedro Lopes. We'd also
like to thank the CHI community for a Best Paper award: we appreciate the
gesture enormously. So, I'll be talking about how this user
is actually able to feel the temperature of the virtual scene
using a low-powered device that we created specifically for this effect.
This work is largely motivated by how virtual reality has become increasingly realistic.
Here, I whipped up two different scenes and
they're both fairly photorealistic already.
Alongside this kind of visual realism, industry and researchers have been
exploring other modes of realism, like touch, force feedback,
odors, or even rendering the temperature of virtual scenes.
I'll be going over how researchers in industry have been doing this kind of
thermal feedback for these devices. The most evident way to do this is via
heat lamps or air conditioning units, which essentially change the temperature
of a large volume of air. VR arcade setups, like the VR Sense,
in fact use this kind of technique.
Unfortunately, these require a lot of energy.
So it runs contrary to what virtual reality is moving towards,
which is mostly untethered devices. In fact,
VR's biggest impact and promise is in home setups or fully portable
setups like the Oculus Go (on the left)
and the Oculus Quest (on the right).
Nowadays, researchers have actually been miniaturizing these kinds of
thermal feedback and even putting some devices inside of the
headset to create these sensations Often times, researchers use
something called thermoelectric materials or peltier elements.
What's nice about these, is that they can be really small, and the way they
function is, when you put a current through it,
it creates a temperature difference across the surface: one side gets hot and the
other side gets cold. Unfortunately, to sustain the temperature difference,
you need an equivalent amount of power.
So, they can get small, but they usually remain tethered.
Let's take ThermoVR for example. If you wanted to create a 10-minute
full cold scene for the VR environment, you would have to take 10 of
the peltiers on the device, put each of them at negative 3 degrees celsius,
and then sustain that for 10 minutes.
Each one would take about 0.5 Ah, which means the whole thing is going to
take approximately 5 Ah. And that's already 1.5 times your
phone's battery life. So you're starting to get the problem
that's happening right now and our goal is basically: can we
actually create these sensations without using lots of power,
which probably means that we can't use a technique that's bounded by physics.
In fact, our approach relies on a thermal illusion,
in which we chemically actuate our senses. For this, we focus on the
chemical senses that we have, in particular the ones in our nose.
Now, most people are probably familiar with the tongue
for tasting or the olfactory bulb for smelling things,
but there's a third nerve in our in our face called the trigeminal nerve.
Most people aren't familiar with this, but they actually experience at the
same time as tasting and smelling things. So if you've ever had something spicy or bubbly or refreshing:
those kinds of sensations are actually the trigeminal nerve or responding.
Now, this nerve responds to mechanical deformation (tactile stimuli) as well as having receptors that
respond to temperature stimuli.
So if you go outside at -10 degrees celsius, in like
the Chicago winter, and you breathe in the cold crisp air. That feeling
is in fact the trigeminal nerve responding, and what's pretty neat about
this is that these receptors not only respond to temperature,
but also to chemical stimuli, which makes you wonder:
Can we actually create all these sensations only with chemical stimuli?
So here we're focusing on two particular receptors:
one that's called TRPV1 that's temperature-gated
aka it starts firing off and responding to things once
the temperature in the tissue is around 45 degrees celsius and above,
which is very hot for the body. And then the TRPM8, which is a cold receptor
that responds to temperatures in the body that's 25 degrees celsius and below.
Now, these two receptors also respond to
things found commonly in plants, like capsaicin in
chilies or menthol in mint. We were interested in seeing if we
could use already available essential oils as well as easily accessible chemical compounds to
create these kind of temperature sensations in virtual reality.
So the first thing we did was
implement a prototype thermal display, which in our case is in
fact an olfactory display. Here we're using a phone to
wirelessly communicate with it and activate it.
The device includes a bluetooth microcontroller for wireless communication,
micro pumps to pump around all the liquids, one milliliter reservoirs for each
compound, and then a vibrating mesh atomizer to create an aerosol.
This lets us do things like: emit
capsaicin to create a warming sensation, and also all we have to do for that is
pump some of the liquid from one reservoir
out to the mesh, as well as do things like
using eucalyptol to create a cooling sensation,
or even output mixtures like capsaicin with a smell so we could do
temperature plus odors.
Now two problems we had to take into
consideration were the following: the first one was that capsaicin doesn't
evaporate at the same temperature as alcohol.
In fact, it crystallizes.
So we couldn't do evaporation, which is a pretty common feature in olfactory displays.
Instead we had to keep everything in
liquid phase, aka we had to create an aerosol
of tiny droplets. To do this, we ended up using a vibrating mesh transducer, which
creates the aerosolization. Second, because we're only using one
atomizer, we run the risk of creating cross contamination between the channels.
You wouldn't want eucalyptol and capsaicin to run at the same time
unless you specifically wanted that.
To avoid this, what we did was give each channel
its own tubing and pump. On top of that, we added capillary tubes
at the end of the tubing (at the contact point for the atomizer).
What this allows is when the micro pump creates droplets, the
atomizer quickly gets the droplet and atomizes it,
so there's not going to be any lingering liquids.
But the capillary tubes have a secondary purposeL
because they constrict the flow so much, there's no backflow.
So if you create a droplet, it's not going to float back into
another channel's tube.
Finally, we also made it so that these reservoirs are pretty easily accessible,
so you can take one out, refill it, or even swap it with another compound.
The device is pretty interesting on its own, in that - when you render
temperatures - it only costs about 0.25 W.
That's a stark contrast to peltier elements, which might which may take
50 W or more
So in the studies, the first thing we were asking ourselves was which
stimulants were best to create these hot cold sensations.
We ended up looking through medical and olfactory literature to find first
three warm candidates, including thymol, a cayenne pepper tincture (which is - in
our case - capsaicin and I'll be referring to it that way from now on),
and then methyl salicylate. Additionally, we chose three cool candidates: linalool,
eucalyptol, and peppermint essential oil, of which the last one has a lot of
menthol in it. We also note that methyl salicylate and
linalool actually activate a mixture of
warm and cool TRP channels.
For a baseline in this kind of study, what we ended up using was the solvents
(what we used to dilute all these compounds, in this case ethanol aka
everclear). So participants inhaled 1 uL
of these compounds, and then had to rate their perceived temperature sensation.
First, we found that linoleum and methyl salicylate both elicited
a large range of different temperature sensations and that lines up pretty well
with the fact that they both actuate cool and warm TRP channels.
We then found that peppermint essential oil and thymol are actually the
strongest perceived temperature sensations.
Unfortunately, they're both not only smellable (so if you if it's in the air you know there's
something that you're smelling), but readily recognizable. So if you smell peppermint essential oil,
you'll probably recognize that it's
minty or that it's probably just peppermint.
Unfortunately, what we're looking for in this case is
a compound that not only elicits temperature sensations,
but also is not very recognizable so it doesn't break the temperature illusion.
Luckily, eucalyptol and capsaicin fit
this criteria very well. Despite being smellable,
eucalyptol was not readily recognizable by most people in the study.
Additionally, capsaicin is odorless on its own.
Finally, people were able to distinguish these kind of temperature sensations
with only 1uL of volume. That's incredibly small meaning we
can use very small quantities of eucalyptus and capsaicin
to create the kind of temperature illusions that we wanted.
We then tested these temperatures in VR. Participants were placed in a VR scene,
either a hot desert or a cold mountain,
and then either presented the baseline (aka the solvent again)
or the temperature-congruent trigeminal stimulus. So in the case of
the cold mountain, it would either be ethanol for the
baseline or menthol for its trigeminal stimulant.
What we found is that the temperature illusion held up.
In the cold mountain case, it was actually much colder
with menthol, and then, in the in the hot desert section,
when you included capsaicin, you actually created largely warm sensations.
So this ended up confirming that these trigeminal stimulants
with visually-congruent environments actually create the thermal illusion.
So we ended up taking these results
alongside our prototype, and created a full experience for them.
Here, a user is warming up at the furnace inside of a random cabin in
the woods. We render the warmth of the furnace
via capsaicin, which activates the TRPV1 channel
in the trigeminal nerve, creating an odorless warming sensation.
Then, because of a storm,
a power outage comes, which cuts off all the electricity and the heat.
Now, because of the cold winds are coming in, we end up emitting eucalyptol
to create a cooling sensation via the TRPM8 channel
of the trigeminal nerve.
Because it's cold, the user goes outside to try and find the emergency
electric generator.
And because it's colder outside we, actually emit eucalyptol more frequently.
We then find the the emergency generator,
switch it on, which then provides a bit of heat so we add in some capsaicin
and go a step further and throw in a rubbery odor
to create the sensation of burning fuel.
Our device is doing not only temperature,
but also mixing in odors.
So now that the power is back on, they're going to go back inside and stay warm.
On their way in, they turn around and close the door.
At this point, we stop the eucalyptol, since there's not much there's not much
cold air coming in anymore, and instead start emitting capsaicin again.
At which point they can go warm up at the furnace.
Now, to conclude, we introduced a new way of creating temperature sensations in
virtual reality using an illusion that's about
52 times more efficient than peltier elements.
However, it is still illusion, so right now the range of sensations is
quite limited, but this opens up a new venue of researching
synthetic compounds for these temperature receptors.
For example, insulin is a synthetic compound
for the TRPM8 cold receptor, and is about 50 times colder than menthol, and
has no odor. Another consideration is just that these
kinds of sensations are tied directly with the user's breathing,
and users might sometimes unconsciously alter their breathing to avoid exposure
to certain compounds. At the moment, the sensation is pretty
much limited and localized to the face, but several participants actually
expressed that it spread to the chest.
Finally, this does allow a very neat and simple upgrade for olfactory
displays to render not only odors, but also temperatures
and this extends beyond VR contexts to mobile and
augmented reality ones as well.
Now while this illusion has a lot of promise,
we're excited to explore it at other locations to create richer thermal experiences.
So I'd like to thank my collaborators again. First my my colleague Steven Nagels, and then my
advisor Pedro Lopes. Thank you for your attention, and I'll
answer your questions now.