Greetings everyone! My name is Yatharth Singhal and\h today the work that I will be presenting is titled\h\h
Mid-air Thermo tactile feedback using Ultrasound\h Haptic display. This work was done in collaboration\h\h
with Haokun Wang, Hyunjae Gil, and professor\h Jin Ryong Kim from Multimodal Interaction Lab\h\h
at the University of Texas, Dallas. In our day-to-day\h life we see many examples of thermal feedback\h\h
and it is an integral part of human lives.\h We believe that in order to create more immersive\h\h
applications this modality should be integrated in\h VR as well. In a VR application, it can be used to\h\h
enhance realism, provide perception of virtual\h objects and also as an ambient communication\h\h
channel. There are two broad classifications\h on how thermal feedback can be incorporated\h\h
in VR: Contact-based and Non-contact-based methods.\h Contact-based methods often involve use of peltier\h\h
devices or gloves to provide thermal feedback on\h user skin but they require the use of variables\h\h
which is not as intuitive as bare hand interaction\h Non-contact-based methods provides the bare hand\h\h
intuitiveness but possess the difficulty\h of controlling thermal cues in a 3D space\h\h
That brings us to our research questions. Can\h we provide thermal cues to bare hand while\h\h
controlling their position in 3D space? Can we\h create and control thermal cues in mid-air? and\h\h
finally do these localized thermal cues bring\h enhancements in VR experiences. It is known that\h\h
thermal and technical perceptions are perceived by\h different receptors on our skin. Mechanoreceptors\h\h
perceive tactile sensations while thermal\h sensations are perceived by Thermoreceptors.\h\h
Because of this these signals are processed\h in different regions of human brain as well.\h\h
Tactile signals are processed in the Somatosensory\h cortex while thermal signals are processed in the\h\h
Insular cortex. This enables humans to perceive two different modalities at the same time. We adopt a\h\h
perception based approach where we provide unified\h thermal and tactile feedback in mid-air. This\h\h
thermo-tactile feedback is composed of localized\h tactile feedback and global thermal feedback.\h\h
Thermal and tactile feedback are stimulated on\h the same area and therefore they are perceived\h\h
together without masking each other. We designed\h a prototype to present thermo-tactile feedback\h\h
in mid-air. It consists of an ultrasound display on\h it's base to provide local vibrotactile feedback,\h\h
two heat modules to propagate heat and provide\h global thermal feedback. The prototype itself\h\h
is a dome-shaped chamber that retains heat for\h better temperature control. The system also has a\h\h
hand motion tracker mounted on the top which\h can help provide sensation on user's palm.\h
Our prototype can provide a maximum\h thermal feedback of 54.2 °C.\h\h
It can also provide tactile feedback up to 3.43 mN. We also confirm the\h\h
stability of the system with a mean error rate\h of 0.25% in temperature fluctuations.\h
We conducted three user experiments to verify\h our hypothesis. Our first experiment measures\h\h
perception threshold for thermo-tactile feedback\h in mid-air. Experiment 2 is conducted to see\h\h
if thermal feedback affect participants ability to\h identify and distinguish between tactile patterns.\h\h
and finally, we propose some VR application\h of thermo-tactile feedback and study if users\h\h
prefer thermo-tactile feedback condition over\h other feedback conditions. Experiment 1:\h\h
Measuring thermo-tactile thresholds. This\h experiment was conducted to estimate\h\h
mid-air thermo thermotechnical thresholds\h for warmth and pain detection in mid-air.\h\h
Warm Detection Threshold or WDT is defined as the\h the temperature at which users start perceiving warm\h\h
sensation and the Heat Pain Detection Threshold\h or HPDT is the temperature at which users start\h\h
feeling thermal pain or discomfort. For this\h experiment we had 12 participants with a mean age\h\h
of 25.2 years. We followed the one up one down\h staircase method to calculate the thresholds.\h\h
Participants were exposed to\h the sensations for seven seconds\h\h
and after that, they were asked to reply "Yes" or "No"\h depending on if they felt warm or thermal discomfort.\h\h
We continued the experiment for six reversals\h and then took the mean of all the reversals.\h\h
This was repeated twice for each threshold the\h graph here shows WDT and HPDT calculations for\h\h
one participant, we found that the mean WDT was 32.8 °C and the mean HPDT\h\h
was 44.6 °C. These results are in\h range of thresholds for contact-based methods.\h\h
Experiment 2: Tactile pattern identification. We\h conducted this experiment to study if thermal\h\h
feedback interferes with the ability to\h identify technical patterns in mid-air.\h\h
12 new participants with a mean age of\h 24.2 years took part in this experiment.\h\h
We designed three tasks to identify number, shape, and direction of tactile points. Within a task,\h\h
a pattern was randomly selected and\h presented with either No Thermal\h\h
or Thermal condition. The No thermal condition was\h at room temperature which was set to 23 °C\h\h
and the thermal condition was set at 44\h°C. Each pattern was repeated 10 times\h\h
and the participants had to identify\h the pattern being presented to them.\h\h
We calculated the identification accuracy for thermal and non-thermal conditions. We found that\h\h
the overall identification accuracies were quite\h similar for both non-thermal and thermal conditions.\h\h
98.1 percent for non-thermal and\h 97.2 percent for thermal condition.\h\h
This proves that the thermal feedback does not\h affect tactile pattern identification in mid-air.\h\h
Experiment 3: VR Applications. With this experiment\h\h
we proposed some possible application\h of thermo-tactile feedback in VR.\h\h
We also studied the preference of thermo-tactile\h feedback in comparison to other modalities.\h\h
16 participants with a mean age of 25.6 years\h participated in the study we developed two VR\h\h
scenes Camfire, where users can see and interact\h with embers from the fire and the Water Fountain\h\h
scene where the hot water coming from water fountain was presented.\h
We carefully created customized tactile\h sensations that matched with the visual feedback.\h\h
In addition to the audio-visual modalities,\h we presented four feedback conditions:\h
None, Only Thermal, Only Tactile, and Thermo-tactile.\h Participants were presented with these conditions\h\h
in a random order and were asked to rate based\h on Immersion, Enjoyment and Satisfaction of the\h\h
condition on a VAS scale. We found significant\h differences between thermo-tactile condition\h\h
and all the other conditions. It can also\h be seen that thermo-tactile condition was\h\h
the most preferred modality for all the three\h measures, across both the scenes. Applications:\h\h
Here, I present a short video clip exhibiting\h the process and VR scenes that we developed.
Finally, to wrap up we proposed a perception-based\h approach to provide unified thermo-tactile\h\h
feedback in mid-air. We also show feasibility\h in providing thermo-tactile feedback\h\h
with varied shape, size, and frequency in a 3D space.\h We measured thermo-tactile thresholds (WDT and HPDT)\h\h
in mid-air and found that they are\h in the range of contact-based methods.\h\h
We observe that thermal conditions do not affect\h haptic pattern identification. And finally, we found\h\h
that people find thermo-tactile feedback more immersive, enjoyable, and satisfactory. Thank you\h\h
so much for listening to the presentation\h and please feel free to ask any questions.