Hi. Thank you for coming to see my presentation
I’m Chaeyong Park, a PhD student at POSTECH, South Korea
The title of the paper is
“Visuo-haptic crossmodal shape perception model for shape-changing handheld controller bridged by inertial tensor”
Our research was conducted by my co-author, Jeongwoo Kim, and my advisor Seungmoon Choi.
In virtual reality, handheld controllers are basically input devices,
and provide various haptic sensations to improve the immersive experience
But, typical handheld controllers have fixed shapes and sizes,
so they cannot represent the perceptually similar shape of various virtual objects
Thus, the user experiences of visual and haptic data are not consistent
To this end, to provide visual-haptic congruent sensory experiences,
researchers proposed shape-changing controllers as a solution.
but, we do not have any general computational models
that explain the human perception of shape-changing handheld controllers
In this paper, we build a visuo-haptic crossmodal shape perception model
for shape-changing handheld controllers
to provide congruent visual and haptic multisensory experiences to users in VR
Before introducing my work, I briefly explain dynamic touch,
which is a pivotal concept for our research
Dynamic touch is the kind of touch mode
Humans perceive the objects’ shape information, such as length, width, heaviness, volume, and orientation,
through minimal body movements like holding and wielding
In dynamic touch, the moment of mass distribution significantly affects on perceiving the shape information
The moments of mass distribution include mass, static moment, and inertia tensor
The static moment is related to an object’s center of mass
The inertia tensor is related to the rotational inertia of each rotational axis
The inertia tensor has three diagonal terms called the moments of inertia
and six off-diagonal terms called the product of inertia
But, because it is hard to consider all physical variables to build a perceptual model,
we conducted a computer simulation to find the important variables for shape perception
We assumed a handheld object with mass distributions projected onto a half-plane
by varying the length and angle between two cylindrical parts
Then, we compared elements of moments of mass distributions
with correlation coefficients and PCA
As a result, we choose the primary moment of inertia, called MOI,
and the product of inertia, called POI, as the important variables.
These are examples of MOI and POI of different shapes of cylindrical rods
by varying 1D lengths,
2D area or asymmetry,
and both of them
Then, we conducted two user experiments
to confirm the perceptual importance of the two variables
We investigated the human discrimination performance of the length and asymmetry of handheld objects
Participants wielded a physical object
and evaluated its perceived length in experiment 1 and asymmetry in experiment 2
The experiments followed methods of constant stimuli
that can derive the psychometric function of the perception
Furthermore, we can derive some thresholds and the corresponding Weber fractions
The results of experiment 1 showed that
the MOI of a physical object has a dominant effect on the perception of the object’s length
We investigated the perceived length of objects from short to long
And the Weber fraction of length perception was very low about 7%,
which means that people can discriminate a 7% difference in MOI
regardless of the object’s actual length
In Experiment 2, POI had prominent effects on the asymmetry perception
with significant interaction with MOI
There are the entire psychometric functions and the sensitivity of POI decreases as MO increases
Then, we built a visuo-haptic crossmodal shape perception model
by conducting the third user experiment
The input variables are the properties of a physical handheld object,
and the output variables are the properties of a virtual object in VR
It means that this model suggests the best visual shape of a virtual object
that matches the perceived attributes of a physical object in the real world
The experiment contained various shapes of objects with different MOI and POI
Participants interacted with a physical object and then made a virtual object
until they perceived visual and haptic stimuli as similar
Then, we obtained a regression model that explains haptic to visual shape perception
This figure visualizes the regression model for crossmodal shape perception
There are large differences in the aspects depending on the range of MOI
In addition, in the case of physical objects at POI is 0,
their virtual matches showed increasing POI as MOI increased
This behavior indicates a strong interaction between MOI and POI, especially large MOI
Furthermore, in actual applications, we can easily compute the physical configuration
of a shape-changing controller that matches the visual shape of a virtual object
using the model’s inverse form
Finally, we conducted a summative user study to assess the validity of our perception model
The experiment consisted of two phases
The first is visuo-haptic crossmodal matching, and the second is assessing user experiences
For this experiment, we chose 5 different shapes of virtual objects
and implemented corresponding physical objects based on the perception model
In Phase 1, participants interacted with one physical object
and matched the shape of the virtual object among 5 virtual objects
Participants identified a haptic object from a visual object with about 87% accuracy
In Phase 2, participants interacted with five virtual scenes
Then, they evaluated user experiences under three haptic conditions:
the best-matching object predicted by our shape perception model,
a mismatched object, and the original controller of the head-mounted display
Our visual-to-shape matching method improved the user experience for all cases
The result said that to increase the user experiences, a visual and haptic matching environment is essential
The contributions of this work are
finding effective physical variables for shape perception including MOI and POI,
building regression functions for visual-to-haptic crossmodal shape perception in VR,
and evaluating the user experiences of the perception model
This is the end of my talk
Thank you for listening