Hello everyone, my name is Sebastian Günther from the Technical University of Darmstadt,
Germany, and today I present you our work where we investigated the interdependency of visual and on-body
thermal feedback in Virtual Reality, called Therminator.
As we all know, Virtual Reality is becoming more realistic and has an ever-increasing
presence for users. However, those new levels of immersion are
nowadays still mostly attributed to an increasing quality of visual and auditory details.
Further, recent works extend the immersion even more by adding haptic components, such
as vibrotactile feedback or similar mechanical forces applied to a user’s body.
While all of those features indeed leverages Virtual Reality to the next level, there are
still other aspects which are not fully explored yet, such as Thermal Feedback.
So, you may ask: Why do we need thermal feedback? Well, the most obvious reasons are of course
to simulate warmth or coldness. However, we also need to consider such temperature
changes on different scales, such as environmental effects or thermal object properties like
a hot mug. And last but not least, temperature can evoke
emotions and well- or malbeing through social experiences.
Okay, so before we can apply thermal feedback to users, we need to have a look at some design
considerations. First of all, we have to take care of the
temperature range. Hereby, we need to consider the neutral skin temperature between 30 and
36 degrees celsisus. However, the range has to be limited since
we do not want to cause any pain and take care of thermal pain thresholds which lie
at around 15° Celsius as a lower boundary, and around 45 to 52 degrees as upper boundary.
Besides the temperature, we also need to consider different anatomical properties of the human
body, such as different shapes for different body parts. For example, a straight leg, convex
abdomen or a bent spine. Further, as humans differ in size, body parts
are also not equal. For example, the upper leg is almost twice the size as the lower
arm and can even vary between individual persons. And last but not least, we do not want to
restrict any movements and have to consider varying degrees of freedom of different body
parts which can have multiple joints and are able to bend or stretch.
In summary, the requirements for thermal feedback are a good thermal conductivity, adapting to shapes
and sizes, and it needs to be flexible at body joints.
As thermal feedback is of course not completely new, related approaches often rely on contact-based
solutions, such as thermoelectric Peltier elements.
Those have very good thermal properties and also have the advantage of being small, lightweight
and can provide very localized feedback. However, due to their nature, they tend to
have a high-power consumption and can restrict movements through their rigidity, especially
when stimulating larger areas. In contrast, there also exist non-contact-based
solutions which use external heating units or fans.
Those also have good thermal properties, but also the advantage to not hinder any movements.
However, since they are not directly attached to the body, it is not trivial to actuate
specific body parts while a user is moving.
So, to overcome those disadvantages we thought: Why not using liquids as medium to transfer
temperature?
Liquids, have the advantage to have good thermal properties and can be used very flexible.
In particular, we had a look at water which has a good thermal conductivity of 0.59 Watt
per meter times Kelvin. Of course, liquids need to be inside some
container to be actually useful, so we decided to take tubes which we can adjust, bend and
where a liquid with different temperature can flow through.
For this, our tubes had to have a similar thermal conductivity as water and allows for
enough flexibility which we found in tubes made of polyethylene of raised temperature
resistance, short PE-RT tubes.
Using such tubes allows us to have different arrangements and numbers of interconnected
sub-tubes in grill-like patterns to actuate small or large regions of the body.
Further, we can reshape the tubes to adapt to different body parts and also are able
to vary the length of each tube. As a proof-of-concept, we decided to design
two actuators. The first can actuate the upper and lower
arm. Here, we use eight sub tubes which are connected through flexible PVC connectors
to avoid movement restrictions at the elbow joint.
Second, we designed an abdomen actuator which aims to provide thermal feedback to a more
flat part of the body. Finally, both then are connected to our system’s
water cycle.
So, how does it look like? As you can see here, we have the PE-RT tubes in red which
are connected by very flexible transparent PVC tubes.
In the current prototype, we use a flow rate of 40 ml per second which resulted in a temperature
change of around 2° C per second as visible in the thermal camera view on the left.
For changing the temperature of each actuator, we mix a hot- and cold-water source using
a thermal mixing valve. An electronic magnetic solenoid valve is used to control the actuation.
We then measure the flow rate and inner temperature of the liquid and are able to adjust the flow
rate through a pressure regulator. Then, the liquid reaches two additional magnetic
valves which controls the flow to the abdomen and arm actuators.
After having our system ready, we implemented three example applications.
In the first, the user is a pirate on an island and needs to find precious gems. Those can
be found in the water which will cool down the user’s arm when reaching for them.
In the second application, a user plays a snowball game and everytime he gets hit by
a snowball, the abdomen cools down quickly and slowly returns to the neutral temperature
after some time. In a final application, we simulate an emergency
task where the user has to put off fires in a building. The more flames are extinguished,
the cooler the environment gets. - Okay, after this, we thought, why should
we stop here. Because there is something we still can do…
… as Virtual Reality can simulate unfamiliar situations, such as having a hot burning ice
cubes.
We asked ourselves: What happens if thermal and visual stimuli do not match the temperature
expectations of the user? For example, on the left, snow turns warm, and on the right,
the fire turns cold.
Therefore, we defined three research questions which we wanted to investigate.
First, how does the interdependency of thermal and visual stimuli affect the perceived temperature?
Second, how do the thermal and visual stimuli affect the involvement of the users?
And third, we asked how do the thermal and visual stimuli affect the comfort of the users?
To answer those questions, we first had to define the thermal stimuli.
Hereby, we used the neutral skin temperature at 32.5° C as starting point.
We then increased the temperature in 5° C steps until we reach 42.5° C, so we stay
below the upper pain threshold. We also did the same in the opposite direction
and decreased the temperature in 5° C steps until reaching 22.5°C which is just above
the lower pain threshold, resulting in 5 total thermal stimuli.
For the visual stimuli, however, we had to identify meaningful visualizations.
As Fenko et al. stated, objects convey different expectations of their temperature based on
two factors. First, a literal or physical meaning which
conveys the actual temperature. And second, a figurative meaning that describes
temperature by learned experiences. To do so, we interviewed 7 HCI experts and
asked them for visualizations, objects, and other things connected to temperature which
they should rank from very cold to hot. Afterwards, we collected the answers, clustered all items
and counted the common elements.
This resulted in the following five visual stimuli which we applied as visualizations
for the arm, as well as for the abdomen. A very cold snow visualization.
A cold raincloud. No visualization as neutral visual stimulus.
A warm heating-lamp. And a hot fire close to the user’s body.
For the procedure, I don’t want to go into much detail.
Basically, we used a within-subject design with 25 participants.
And provided all of them with a longsleeve while sitting on a chair and wearing our actuators
beforehand. Each participant then either started with
the arm or the abdomen. During each condition, we combined each thermal stimulus with each
visual stimulus and exposed each combination to the users for about half a minute. Afterwards,
we reset both to their neutral stimuli and asked to answer a questionnaire in VR which
I will show you in a second. This was repeated for a total of 50 combinations.
At the end, we asked for additional qualitative feedback through a final questionnaire and
informal interviews.
As mentioned before, after each combination of the stimuli, we asked to fill out a questionnaire
with the following four questions. How do you rate the thermal sensation?
How do you rate the thermal comfort? How much did the visual aspects involve you?
And, how much did the thermal aspects involve you?
In the following, I will present you our main findings. However, if you are interested in
more details, especially in lots of numbers and statistics, I would like to invite you
to have a look at our full paper.
Okay, lets dig into the results. As a first main result, we identified that every thermal
stimulus always overrides the perception of visual stimuli.
So, for example, if we have a fire visualization combined with a cold stimulus, it still feels
cold as you can see here where a lower number basically
means a lower perceived thermal sensation, and a higher number means a higher perceived
temperature sensation. This was true for all other visualizations
and the participants were always rating the applied thermal stimulus as main factor for
their temperature perception.
However, if we take a closer look to the numbers, we can see that the visual stimuli still have
marginal effects on the temperature perception. For example, for a neutral thermal stimulus,
we see a significant higher rating while being exposed to a fire visualization compared to
a snow visualization which is also indicated by differences
in the minimum and maximum ratings of the participants.
One interesting side-effect we could observe, was an unexpected small shift for the rain
visualizations which resulted by some participants describing it as a warm shower instead of
a cold raincloud.
Regarding the involvement, we could identify that congruent stimuli increase the involvement.
As you can see here, a neutral visual stimulus – so no visualization at all – resulted
in no involvement as expected. However, similar as before, a neutral thermal
stimulus is affected by the shown visualizations as you can see here.
Taking a closer look at matching stimuli, which we can find at the lower left and upper
right parts of each graph, we can clearly see significant higher involvement ratings
as compared to non-matching stimuli which were often described as uncanny or even uncomfortable.
Last but not least, with regards to the participants’ comfort, we found that the level of comfort
highly depends on the applied thermal stimulus. As you can see here, our analysis revealed
a significant higher comfort for neutral and slightly warmer temperatures compared to colder
stimuli. As a result, the overall consensus was that
temperatures generally could even be warmer, but should not get colder as more chill temperatures
were perceived as uncomfortable and more intense.
Before concluding my talk, we - of course – identified some limitations.
First of all, we only investigated the abdomen and arm and need to identify how other body
parts will perceive thermal stimuli provided by our system.
Also, we only combined one visual and one thermal stimulus per condition.
However, what would happen if we have multiple stimuli combined at once or even actuate different
body parts with different stimuli at the same time.
And lastly, while not described as restricting the users more than the HMD cable, our system
currently relies on an external source which needs to pump the liquids to the actuators.
Hence, we need to find reliable ways to increase the wearability, for example by having smaller
amounts of liquids and thermal devices in a wearable backpack.
I hereby want to conclude my talk where I presented you Therminator.
I presented you a set of concepts and design considerations,
A prototype along with three example applications, And a systematic evaluation investigating
the interdependency of thermal and visual stimuli..
.. where we found that thermal stimuli always override visual stimuli,
.. congruent stimuli increase the involvement, .. and the comfort depends on the thermal
stimulus.
I would like to thank you for your attention. If you have any questions or comments, feel
free to contact me directly.
Also, if you are interested, check out the other great CHI ’20 talks of my #TEAMDARMSTADT
colleagues from the Telecooperation Lab.
Thanks, and stay healthy! <3