Hello everyone, I am Yeo from University of St Andrews
This work is in collaboration with many co-authors
from different insitutes as shown here
Today I will talk about JINSense
The first motivation: is smartglass the future?
Actually, I am not sure about that.
But we all know that big companies are working on smartglass nowadays
So, maybe it will become the next dominant form factor in the near future.
And when that happens, we will need a better way to interact with these smartglasses.
Next is about midair gestures.
Actually they are not new
and they are already included in many smartphones
such as Pixel 4, LG and Huawei Mate series.
Some people might say that it is just a gimmick
but I think it can be quite useful sometimes
for example, when your hands are dirty
Especially, during this covid era
we shouldn't be touching things outside in the public
So, we will need this kind of touchless interaction
Therefore, in this work, we propose JINSense
It is about exploring a new sensing technique for device equipped with Electrooculography sensors, EOG.
We repurpose the EOG sensors, embedded inside JINS MEME smart eyewear
as shown in the figure here, you can see there are
three electrodes, one on the nose bridge and two on the nose pads
it was originally designed to detect eye movement
however, we repurpose it to detect midair hand gestures, to sense human proximity, rubbing action and to differentiate materials.
So this new found sensing capabilities enable a various type of novel input and interaction scenarios for such wearable device
So let's look at our setup here,
I leave the glass on top of a desk, and I visualize the raw data on the screen
and when I make hand gestures such as swiping, push and pull or rotation
you can see that the raw signals are changing, and they are also quite unique
So, we believe that using this, we can detect midair hand gestures, and we made an initial prototype
First, we stream the raw signals from the JINS MEME device to a laptop using Bluetooth
Now let me show you the prototype
You can see that when I swipe, it can detect correctly left or right gesture, and also vertical gestures, either up or down.
Besides gestures, we also observe some very interesting phenomena, especially for the plastic bottle
So, when I was trying plastic bottle, maybe on top of the glass, it actually generates a higher amplitude signal than using the hand
And another more interesting phenomena is when I try to rub the plastic bottle, from a distance almost like one meter away
You can see on the screen that it is still detecting the action and it generates a unique signal
So, I think we can detect different kind of material and also the rubbing action.
Next is about water ... sensing
So, when I fill the bottle with water and try to wave it on top of the glass
actually it generates a small amplitude signal
but when I remove the water, it causes a higher amplitude signal
stronger signal without the water
So, Ithink we can also use this to detect whether there is water inside a bottle or not
Let me talk about more potential applications and use cases
we can interact with IoT devices like before going to bed, a person usually removes the glass and put it on a bed side table
and when the alarm rings, he can just wave to dismiss the alarm
Or, a user might fold the sunglass and put it in front of the pocket or the collar hanging like this
so the user can perform midair hand gestures to interact with the smartglass
Also, it can become a multipurposes smart sensor for material recognition, water level detection and dryness detection.
In the future, we want to explore more use cases that are uniquely enabled by our technique
including touchless, discreet and convext-aware interaction
We want to implement and of course to evaluate and study it
In conclusion, we have explored new ways to exploit and repurpose existing EOG sensors to achieve new sensing.
The key takeaway here is there is no need to add new sensors to a device, and it is possible to reuse existing sensors.
Thank you very much.