Hi, I’m Ching-Yi Tsai from National Taiwan University
presenting AirRacket:
Perceptual Design of Ungrounded, Directional Force Feedback to Improve Virtual Racket Sports Experiences.
This work is conducted with my co-authors
I-Lun Tsai, Chao-Jung Lai, Derrek Chow, Lauren Wei
as well as our two advisors Lung-Pan Cheng and Mike Y. Chen.
Racket sports, such as ping-pong, badminton, and tennis
are some of the most popular virtual experiences.
For example, Wii Sports is one of the all-time best-selling games with more than 80 million copies sold.
Also, recent racket sports games have been introduced to Virtual reality,
and even includes online gameplay to provide a social and competitive multiplayer experience.
In the real world, These experiences include directional impact force on the racket,
which includes the haptic sensation of tactile, kinesthesia, and proprioception.
But given racket sports’ high mobility, current experience is usually provided in limited fidelity
for example, controller vibrations
Directional, ungrounded force feedback technologies such as air jets and propellers-based solutions,
have the potential to provide a more realistic haptic experience
as they provide an actual directional force in handheld and wearable form, enabling player mobility.
However, the current maximum force magnitude of ungrounded force haptic devices are around 5N
, which is 2 orders of magnitude smaller than peak force in real-world racket sports
which, taking tennis for example, can exceed 400N
This drastic difference in force magnitude makes haptic feedback for racket sports especially challenging to design.
And how to render impact force with a larger magnitude is the main limitation we are addressing in our project.
In AirRacket, we explore a solution based on haptic perception.
Specifically, we verified and utilized that with longer impact duration, users perceive larger impact force magnitudes.
And to fit this insight in applications, we created devices for a set of racket sports using compressed air propulsion jets
Our propulsion force system consists of a pneumatic control system that modulates the air pressure from the air supply
and thus the output force magnitude.
With two solenoid valves, we control the on-off of force output on1dof direction,
which is for forehand and backhand strokes.
And three handheld devices for virtual ping-pong badminton and tennis,
are connected to the pneumatic control system
mimicking the real weight and wielding sensation of their real-world counterpart.
We use opti-track mocap system to track them in VR
, but versions with vive tracker can be found in our open-source project folder.
The maximum force of these devices are 3.2N
, with a 22 ms rise time to reach it.
while the noise level is 77db, similar to an indoor tennis hit.
The max frequency of continuous impulse is 40Hz at 1N and 25Hz at 3N.
After creating these devices, we continue to the part of our main user study.
First, we verify the assumption and question that
whether impact force duration affects the perceived force magnitude.
To answer this question, we conducted magnitude estimation study to
understand the relationship and scale between impact force duration and its perceived magnitude.
In this study, participants rated target forces of various durations and intensity with verbal numerics,
where a constant force with fixed duration serves as the reference.
The haptic stimuli are given without visual or audio feedback
, and our experimental duration range ranged from 50ms to 350 ms
, while the experimental force magnitude ranges from 0.5N to 2.5N.
The result was normalized by the reference force, and here’s the average mean across 12 participants.
Participants reported significantly higher perceived force magnitude as force duration increased.
For example, the perceived magnitude of impact with 350ms is perceived to be 4 times stronger than its 50ms counterpart.
Also, the perceived increase was significantly higher (p<.01) for long vs. short devices for all durations.
We further analyzed the result with logarithmic transformation on duration,
which finds a strong linear relationship between the log of duration and perceived force intensity.
This result is in accordance with the so-called power law in previous magnitude estimation studies.
The same power function relationship has also been found previously in audio stimuli’s duration and its perceived loudness,
, also visual stimuli’s duration and its perceived brightness.
To put the finding in practice we construct four mapping models to map expected force magnitude in the real-world to our system output.
These models are with different concepts, including duration varying, magnitude scaling, and their combination.
Two of our models change the force magnitude only,
while in the first baseline models, we match our output peak force value to real-world intensity,
, but render system max when we can’t attain it.
The second “scaled” model is a linear interpolation across our system output,
mapping the system max to the users’ maximum relative swing motion.
In both models, the output impact duration is constant.
In the next model, we started to explore the concept of duration variation,
in which we render constant, force magnitude at system max
but with varying duration corresponding to the users’ relative swing motion.
The last model is the combination of the previous two concepts,
where we were scaling force magnitude with varying duration at the same time.
To find the proper duration range that we can modulate and scale with, we conducted two supplementary studies.
One is a participatory-design study to finds the acceptable duration variation range that we can utilize,
which we found users’ longest acceptable duration can be 70 times longer than real-world impacts’ contact time (5ms).
The other is an absolute detection threshold study to find the minimum perceivable force used in our force scaling models,
although not in this video details of these two studies can be found in our paper.
Then we conducted a final evaluation of these candidate designs.
We recruited 24 participants to the ping pong, badminton, and tennis VR scenes, to experience and compare these models
In each virtual scene, we have three different ball-serving speeds as well as aiming targets.
Participants were asked to hit the ball in the aiming targets to make sure their swings are within a similar speed range.
We conclude the result in three main findings.
First, the average Likert scale on perceived haptic realism and preference shows duration-varying method can improve users’ impact experiences.
And for ping pong and badminton, we found significant improvement for the combination of force magnitude and duration variation.
These results showed that given the exact same system limitations, perceptual design, and especially multi-variate perceptual design,
can significantly improve the user experience vs. the baseline or physics-based model,
benefited from finer and more distinguishable feedback in the gameplay.
For tennis, no statistical significance is shown in Likert data.
But System Max+Duration Variation was preferred by 46% of participants,
, while participants have commented that "For tennis, the stronger the force, the more realistic it felt, so System Max + Duration Variation was best.”
, which might indicate that such a model is likely to work better for experiences that are consistently far above the system capability.
Besides, a few of the participants have indicated that a better feeling of feedback intensity helped them adjust the swinging speed
for the next ball to aim at the targets,
which indicates how users' performance can benefit from a haptic system’s overall dynamic.
Before the end of this presentation, I’d like to highlight some discussion points as well as future works.
Compared to prior visuo-haptic approaches, the haptic illusion we discovered is unique in that it does not require visual feedback.
Therefore, it may work especially well in combination with a wide range of visuo-haptic techniques that create impact illusion,
for example, using freezing frames to provide pseudo impact.
Impact sensation is common in every day and VR experiences
such as the egocentric impact of being hit, percussion of drumming, recoil when firing a weapon, or kicking a football,
which all have drastic limitations in force magnitude compared to the real world,
which should also be benefited by our feedback design.
Lastly, while our devices are with 1dof force direction at a fixed point,
a motorized moment arm or more degree of freedom might lead to even more realistic experiences,
to support the haptic sensation of slicing a ball or hits that is off the sweet spot.
Also. additional racket designs, including hockey and racquetball, can be found in our project folder.
Thanks for listening.