Hello, I am Adwait, and on behalf of my co-authors
I will be presenting our work SoloFinger
This is a joint work between Saarland University and the University of Copenhagen.
Oftentimes, our hands hold a wide variety of objects
and manipulate them based on the planned activity.
However, gestural input techniques
while holding objects are prone to false activations
Due to grip adjustment, natural finger movements
and various object manipulations.
A gesture recognizer might misinterpret these as input
leading to false activations
In this work, our goal is to avoid false activation for digital input
input while holding or manipulating everyday objects.
Inspiring prior work in this direction
has mainly focused on holding smartphones
and for smartwatches with delimiting gestures, such as flipping or pressing.
However, these gesture designs would not generalize to everyday objects.
For instance, while holding a coffee cup or using the air pump.
In contrast to these device-specific, delimiter gestures
which by the way, would interrupt the primary activity
if you're holding an object
our approach uses easy & rapid-to-perform
single-finger movements that are compatible
with a wide range of everyday objects
More importantly, it is resilient to false activations
We used our observation
and built two key assumptions
that fingers either remain static
or multiple fingers move concurrently during everyday hand-object actions.
And this creates a sweet spot for single-finger movements
that are unique and not present during everyday hand-object actions.
We call this concept as SoloFinger
a robust and easy gestural input technique
to avoid false activation while holding objects
We confirmed our two assumptions
by contributing peak score metric
which quantifies how strongly a single-finger moved
with respect to other fingers.
In the graph, you may see
that peak score for daily hand-object actions
is extremely low compared to SoloFinger gestures.
The ease of single-finger movement
makes our concept generalizable on diverse grasp types and objects.
Thanks to the dexterity of our fingers,
multiple gesture variations can be created.
For example, a designer can create zigzag gesture
for sharing a photo
and swipe left for rejecting calls.
We performed a series of data-driven analyses
to validate our concept, which you may find in the paper.
To give you a brief idea
we used an existing dataset and compared it to
our collected dataset of SoloFinger gestures.
Our extensive dataset collected using Optitrack
is available to the research community.
We performed our concept validation
with 36 everyday hand-object actions
representing a diverse set, such as pouring wine
reading a letter, writing, and many more.
Through a simple thresholding technique
we found no false activation for 23 actions
in the large dataset of daily hand-object actions.
We discuss the actions wherein false activation occurred
with more details in the paper
and later, we show a more sophisticated classifier
that can eliminate false activation on such actions.
Now let's move on to the deployment of our concept.
While our concept can be realized using various sensing techniques
like objects augmented with high-resolution sensors
IMUs or radar-based approaches
we used a commercially available VR glove
we collected a second dataset with hand-object actions
and SoloFinger gestures.
We used this dataset and trained a multi-class classifier
using random forest
we included 7 gesture variations and hand-object actions
If the held object is known,
we achieved an accuracy of 89%, with no false activations
and if the object is not known
the accuracy is 86% and the false activation rate still remain
low to around 2%
To summarize
we presented a novel concept
that uses only one finger for gestural input while holding everyday objects
Through a series of analyses,
we show our concept is resilient to false activation
And ultimately, we used a commercially available hardware
and trained a multi-class classifier
to show 7 gesture variations can be detected with very low false activation.
Thank you