HI, I'm Takahiro Kusabuka, working at NTT Service Evolution Laboratories.
We proposed IBUKI, Gesture Input Method Based on Brething.
Breathing gesture is like this.
Exhale
Inhale
Double Exhale
Swipe
Circle
Gestures combine of breathing patterns and it's trajectories.
So, First of all, Background.
In order to prevent the spread of infectious diseases.
It is important not to bring viruses into your living space.
It is not good touching your belongings like smartphone, earphones, after touching public fixtures such as a handrail or a vending machine.
So, we proposed input method using intentional breathing, instead of touching the device directly.
Breathing gesture is combination of breathing patterns and it's trajectories.
Next, explain about prototype system.
This is prototype system.
We use mask as wearable computing's platform.
A mask is recommended or required to be worn in various situations to prevent the spread of infectious diseases.
So, this system do not need to wear a new device to the body, like a chest belt type sensor.
Then, about hardware.
A brething sensor and a 9-DoF IMU to estimate the head direction are attached to the mask.
And combine the breathing patterns and trajectories to estimate braething gestures.
This is a microcontroller, it's collect sensor's data and sending to a PC by bluetooth or USB.
In this demo, using USB to sending data.
And, On PC, estimating breathing gestures by recieved data.
This is a breathing sensor.
Inside...
This is air valv.
Air valv, moving as the user breath, and amount of movement is detected by photo reflector.
This is an example of sensor's data.
So, It can detect such as whether a user is inhaling or exhaling,
and legngth, speed, intensity, and number of times per time unit of each inhale or exhale.
We use kShape to do clustering of the acquired time series data.
This is a result of expelement. The recognition accurarcy for one subject was 95.8%.
To estimate direction and trajectories by 9-DoF IMU attached to a mask
This is also recognized by kShape.
This is a result of expelement. The recognition accurarcy for one subject was 93.9%.
Next, demonstrating some applications example.
First application is device operation for listning music
This is music controll application.
Breathing gestures and operations are linking.
For example, exhale is to play music, inhale is to stop music,
swipe is skip music.
♪
Next demo is entertainment, gaming interface.
Breathing gestures and controll gaming is linking.
For example, swipe is weapon change, and exhale is shoot the weapon.
like a VR game.
Next demo is Supporting Voice User Interface.
For using Voice User Interface, We should speak wake word, like a "OK, Google", "Hey, Siri".
Using Breathing gestures instead of wake word.
For example, double exhale gesture is calling Voice User Interface instead of wake word.
Additionally, giving context for Voice User Interface.
For example, double exhale gesture is assigned for adding to shopping list,
double inhale gesture is assigned for searching for cooking recipes.
So, Even if the word "fish" is used in both contexts,
the former gesture will search for recipes for fish dishes.
and the latter will add fish to the shopping list.
That’s all.
We proposed IBUKI, Gesture Input Method Based on Breaghing.
Thank you for listning.