Hi, I am Chieh Tsai from National Chengchi University, Taiwan
I am going to share our work, FingerX.
With the development of VR and metaverse, users experience VR not only in an empty room
but also in a space with furniture or objects in the real world,
so we suppose that real objects could be blended into the virtual world not only via visual feedback,
as in current AR interactions, but also via haptic feedback.
Interacting with virtual and real objects, or even virtual objects augmented by real objects
becomes a trend of VR interactions.
Therefore, we propose FingerX to render haptic shapes and enable users to touch,
grasp and interact with virtual and real objects simultaneously.
To render haptic shapes, previous works move or restrict the hands’ or fingers’ movements
using mechanical brakes and actuators.
However, users have to take off wearable devices
or put down handheld devices to perceive real objects.
Furthermore, these devices move or restrict fingers by pulling the fingers from the back,
which is different from the reaction force applied
from the front of the fingers when grasping or touching objects.
Such a difference is distinguishable and affects users’ VR experiences,
as mentioned in Wireality.
Other devices change their shape to virtual objects
as proxies or props using a pin-based design or airbags.
However, the users cannot perceive real and virtual objects simultaneously.
Therefore, a device rendering haptic shapes of virtual objects
augmented by real objects has still not been explored.
To the best of our knowledge, Touch&Fold is the closest work to ours
and also notices the essential of interactions across virtual and real worlds.
However, it focuses on switching modes to interact
with virtual or real objects alone rather than simultaneously,
and it renders compression and vibrotactile feedback instead of haptic shapes.
We propose FingerX with extendable and withdrawable supports to achieve the goal.
It consists of four extenders and rings on each finger except for the pinky finger,
which is usually an auxiliary finger.
A five-stage scissor lift design is used in the extenders,
and a screw mechanism with a self-locking property is used to actuate the extenders.
The extension height is between 22.7mm and 78.8mm,
so the extension range is 56.1mm.
Furthermore, a ring rotates the extender via a ring gear to withdraw it when touching real objects.
Each ring is placed on the second finger segment close to the fingertip.
When touching real objects, the rings withdraw the extenders
behind the fingertips above the fingernails.
When perceiving virtual shapes, the rings on the corresponding fingers
rotate the extenders to the fingertips,
and the extenders extend to the corresponding heights independently to render haptic shapes.
Therefore, the users perceive shapes with the reaction force from the front of the fingers.
FingerX enables users to interact with virtual and real objects simultaneously in three common scenarios,
including touching virtual objects augmented by real environments, such as a desk,
grasping virtual objects augmented by real objects, such as a bottle,
and grasping virtual objects in the hand.
The users’ fingers penetrate the virtual object and touch the real object or hand at first,
then feel the virtual object “growing” during the extension period, and finally perceive the shape.
We conducted a recognition study for the three scenarios without shape visual feedback,
and performed an additional study with shape visual feedback as in the real VR/AR scenarios.
Common geometric shapes or metaphors of common virtual objects were examined.
A real bottle was used in the second scenario.
Based on the results, it seems that the participants cannot recognize clearly without visual feedback.
They mainly leveraged the height differences among their fingers to recognize the shapes.
In the first scenario, the cuboid with a flat plane was easily recognizable,
but the curved surface of the hemisphere was difficult to perceive.
For the second scenario, the small cuboid was the most recognizable shape
since it stuck out only on the index finger.
However, grasping a curved bottle via the extenders is less stable,
and the rings more easily contacted with the bottle,
which interfered with the recognition.
For the third scenario, the cone was the most recognizable shape
due to its unique narrow tip perceived by the relatively sensitive thumb and index finger.
However, since rings and extenders easily collided with each other in the hand,
and the extenders contacted with the uneven palm,
the shapes in this scenario were the most difficult to recognize.
With the shape visual feedback, the average recognition rate is 76.59%,
and the recognition rates are substantially improved and robust enough.
Generally, the comments are consistent with those in the previous one.
We conducted a VR study to observe
how users interact with virtual and real objects simultaneously in the three scenarios and a real object in a virtual bar.
Three methods, including vibration from a controller,
vibration from five vibration motors on the fingertips,
and FingerX, were compared.
A real bottle and box with a tracker were used in the second scenario,
and a book was used as the real object.
The results show that the feedback by FingerX is more realistic and distinguishable
than that by the controller and vibration on fingers.
For both vibration methods, the participants perceived only notifications instead of shapes,
but when touching the real book, they preferred vibration motors on the fingernails,
which completely did not hinder them.
In addition, some of them mentioned that
including the pinky finger in FingerX could enhance the realism.
In conclusion, we propose FingerX to enables users to touch and interact
with both real and virtual objects simultaneously.
We evaluated the users’ shape recognition ability in the three scenarios,
and conducted a VR study to verify
that FingerX significantly enhances VR experiences in the proposed interactions.
Thank you for your attention.