Welcome ladies and gentlemen to FaceHaptics: robot arm based versatile facial haptics for immersive environments by Alexander Wilberz, Dominic Leschtschow, Christina Trepkowski , Ernst Kruijff and Bernhard Riecke.
In this paper, we explore haptic feedback towards the human face to improve presence and emotional response.
With our system we aim for a flexible solution to enable provision of highly versatile cues and to support the user to walk around in a relatively unencumbered way.
The specific focus of our system is on localized and fully directional cue provision at the face instead of just around the eyes itself, by adding a robot arm to the HMD that can provide stimuli below the HMD.
As such we can provide different types of actuation from different directions towards the face.
Let's take a look at related work of our system.
We deliberately look at the provision of haptic cues - previous systems mainly have attached or integrated actuators insight the HMD or below the HMD to provide wind, thermal cues, vibration, or even suction, inertia and pressure.
Only very limited research has been performed on chemical senses by attaching olfactory displays to a HMD, to provide smell or even to allow people to bite objects - food - to enable food experiences.
There's also some related work in other mechatronic solutions that relate to our robot arm, especially those that have been attached to mobile devices, to the body itself, or even the usage of drones that can provide haptic use towards the users hands.
Most of these systems have certain limitations that we try to overcome with our system, specifically the range and location of actuation, the effect of actuation and the variety of cues, provide a single cue instead of providing various cues in a system over time.
So let's take a look at the system itself. The robot arm is a four degree of freedom robot arm using linear actuators and servos and can be calibrated for face geometry.
Actuation consists of a touch actuator with maximum 5 Newton, wind, a heat element and a spray nozzle that can spray water to watch the face.
The system is operated at 45 degrees per second to limit vibration at the HMD . The overall weight is 405 grams.
However, we do have a counter weight behind the head to balance out the HMD with the system in the front.
We performed two user studies to assess our system.
In the first study we looked at directional wind cue perception by looking at how well users can perceive directional haptic stimuli towards the face and in part looking at how dynamic head movements can improve accuracy of perception.
In the second study, we looked at user experience by basically looking at how adding different face haptic cues affect user experience in comparison to audio-visual stimuli only.
In study one we investigated how well participants could judge the direction of wind. To do so we assessed the effect of head movement and if the face haptics robot arm would compensate for head rotations or not.
We had four different conditions, which you can see in the graphic on the right side. In the first condition, the wind rotated around the head in a static position.
Then we had two oscillating head conditions where the head oscillated by following a butterfly target either with compensated wind or static wind.
And then we had a free head movement condition where users could move their head at free will and at the wind was compensated again.
Directions were grouped into three areas in front of the face
and judgment was done after the cue had finished after 8 seconds where the head had to be pointed towards the direction of wind.
In total there were 24 trials: three wind directions, four conditions and two repetitions.
Now, let's take a look at the results of study one.
The absolute pointing error was lowest for free head movement and highest for conditions with static wind direction.
We found no main effect of wind direction and also no interactions between movement condition and wind direction. The variability in pointing data was similar to absolute pointing errors, and the certainty of judging was higher for free head movement, but no effect of wind direction on certainty was found.
The bottom line is that we found a good directional judgment comparable to
but slightly higher than Nakano et al that had a slightly different setup and a different wind flow shape that affected wind direction perception.
In study two we compared the effect of audio-visual cues
and multi-sensory cues on user experience.
Users were moved over a scripted path with 16 clearly noticeable events,
each associated with a specific type of cue.
During the experiment users wore noise cancellation headphones to
filter out ambient noise.
Each path durated approximately three minutes.
After the experiment users answered questions from a questionnaire about convincingness
and memorability of events, emotional response, awareness of events
and the effect on visual experience, associability of events
and presence.
Now, let's take a look at the results of study two.
Presence, memorability, believability and enjoyment ratings were all significantly higher with multi-sensory cues than with the
audio-visual cues with some variations between events.
We also found that valence and arousal was higher for multi-sensory conditions.
We also found a new type of emotional response namely disgust.
What was also very interesting was that haptic cues had a positive effect on the experienced visual realism of the
environment in comparison to the visual-only condition.
The bottom line here is that improvements are somewhat expected.
The effects seem to be bigger on emotional response
than on presence and the effect of visual realism is interesting but needs to be assessed further.
What's also important to notice is that directionality was rated as important,
but also that cues can be further improved overall.
However, we showed the importance and usefulness of our cues in this study.
Let's take a look at the technical considerations and limitation of our system.
We briefly go over some of the major findings of our system in comparison
to other systems that are connected to the body
as can be seen on the left image, to the surface, like a robot arm mounted on a table,
or systems that are mounted on a drone.
If you take a look at different types of factors,
like weight, operation, speed, accuracy, safety,
but also ergonomics we see that there are differences between the
different systems where our system is quite flexible.
However, there are limitations for example in
weight and operation. However we can provide different cues in a
far wider range than for instance systems that are mounted on a table.
For more details, please refer to the paper.
Now, let's take a look at the outlook of the paper.
There are certain limitations that can be overcome by hardware iterations.
First of all, we would like to take a look at reloading mechanisms
that can reload a specific type of stimuli.
There are some examples in related work that can be used for this.
For example, the rotating head of the haptic Revolver by Whitmire
or magnetic exchangeable heads similar to Topobo.
We would also like to take a look at different types of actuation events, for example to play around with different soft
and hard actuations, and also to focus on other types of events,
for example, fear-inducing events or pain that can be used in therapy or game environments.
Finally, we would like to redesign our system to relocate some of
the hardware towards the top of the head to
alleviate some of the weight issues that we can find in our current design.
In conclusion, we presented a novel haptic actuation system
for versatile cues towards the human face.
We showed how well people can judge directional cues
and how the system affects immersion and user experience.
Thanks for watching and for further details, please refer to our publication.