Hi. I’m Arata Jingu at the University of Tokyo.
I will present “LipNotif: Use of Lips as a Non-Contact Tactile Notification Interface Based on Ultrasonic Tactile Presentation.”
This paper is co-authored with Takaaki Kamigaki, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda.
This study explores the use of the lips as a tactile notification receiver using mid-air ultrasonic haptic technology.
The movie shows a man reading a book and holding it with his hands while listening to music.
He is focusing his eyes and ears on other things and is unaware of a visitor.
LipNotif allows the user to receive information using only lips without sight, hearing, or hands.
LipNotif can convey directional and rough affective information.
For example, with a strong stimulus to the right side of the lips, he can notice that an urgent visitor has arrived on the right side.
The proposed method is based on mid-air ultrasound haptics.
This technology enables remote tactile presentation by controlling ultrasonic transducers to create ultrasonic focal points in the air.
Tactile notification can intuitively convey information such as directions or emotions to users by using various tactile patterns without using their eyes or ears.
To date, many wearable tactile notification devices have been proposed. Most of them are wrist-worn devices.
In addition to that, finger-worn and ear-worn notification devices have also been developed.
Recently, some papers have reported non-contact tactile notification to palms using ultrasonic cues.
Non-contact tactile notification can reduce the physical burden on users, which is one advantage over wearable tactile notification methods.
Previous studies have focused on the palms as the main targets for non-contact tactile notification.
However, when considering non-contact tactile presentation to the hands, there are two issues.
The first is that the hands are constantly used for many other tasks in daily life, such as holding, grasping, and manipulating objects.
The second is that the fingertips and palms, which are the most sensitive parts of the hands, are often facing or closing inward, even if they are not doing anything, making it difficult to stimulate them from the outside.
We focused on non-contact tactile notification to lips.
They have high tactile sensitivity comparable to the palms. In addition, compared to the palms or fingertips, the lips are constantly exposed outward and are less occupied in daily life.
In terms of the balance between tactile sensitivity and occupancy, lips can be another effective place to receive non-contact tactile notification from the outside.
Our previous work investigated basic tactile perception characteristics of lips in mid-air ultrasound haptics.
The lowest tactile thresholds were achieved at the valley-shaped area of the lips in terms of location.
It would be due to the multiple reflections of ultrasonic waves in the valley-shaped structure between the upper and lower lip.
So, in this study, we formed an ultrasonic focal point at the valley-shaped area.
Let me explain how to determine the 3D coordinates of the point to be stimulated.
First, we acquire the 2D coordinates of three main points of the lips by processing the 2D image with a facial landmark detector.
Next, we obtain the 2D coordinates of division points by respectively dividing the straight line from the left commissure to the center and the straight line from the center to the right commissure into some equal parts.
Finally, by combining them with the depth information from the depth sensor, we can calculate the 3D coordinates of each division point.
From these point coordinates, we can select the point to be stimulated according to each purpose.
In the first user study, we examined the feasibility of directional information notification with an ultrasonic tactile cue presentation on the lips.
We defined the direction conveyed by the cue as the direction normal to the local lip plane where the cue was presented.
In each experimental flow, we chose a target point and stimulated a division point facing toward the target point.
Participants looked in the direction they thought was conveyed by the cue and selected a target point in the center of their vision.
As the experimental result, the system could transmit directional information with an average accuracy of within plus-minus 11.1 degrees in the 120 degrees horizontal range without prior mapping.
Considering the humans’ field of view, this result means that the system could transmit the direction with sufficient accuracy.
In the second user study, we investigated the affective effect of different ultrasonic haptic cues on the lips.
As a result, stronger stimuli elicited more unpleasant and more excited emotions than weaker stimuli.
Spatiotemporal patterns showed no significant differences.
LipNotif allows the user to receive information using only lips without sight, hearing, or hands. Also, it doesn’t require users to wear or bring anything in advance.
By combining directional and emotional cues, LipNotif can open new and useful interactions, such as notifications during work, calling in the waiting room, and tactile feedback in automotive user interfaces.
I’ve presented our new work “LipNotif,” which explores the use of the lips as a tactile notification receiver. Please check our paper for more details.