Hello, everyone
Today I will show our recent work called ‚ÄúWatching on Your Phone‚Äôs Back:
Gesture Recognition by Sensing Acoustical Structure-borne Propagation"
Existing works employ
the common built-in sensors (e.g., microphone, accelerometer) to sense tap activity.
They only focus on this coarse gesture due to the limited information obtained from built-in sensors.
Recent study utilizes the sound properties to
enable a fine-grained gestures
such as swiping, scrolling, sensing system on the surfaces of commodity devices.
However, this gesture recognition system relies on the in-air propagation
which can be easily interfered by other movements in the air.
To the best of our knowledge, this is the first structure-borne propagation (SBP)-based
sliding gesture sensing research on smartphones,
demonstrating accuracy and robustness advantage
than alternative approaches based on air-borne approaches.
We propose a novel framework for discriminative gesture recognition.
We design extensive experiments to evaluate our system.
So in this page
We focus on describing the theoretical model
of structure-borne propagation regarding the gesture performing on the back surface of mobile phone.
While the user touches the surface of mobile phone
as shown in the left Figure, the sound signal will encounter the finger.
Part of signal will be directly absorbed by the finger
and other parts will propagate back to the microphone along with the reflection path.
Therefore
the touching location information is related to the reflection path during the whole structure-borne propagation.
The following parts are mainly describes the details to extract
the structure-borne components.
which is highly related to the sliding gestures
For the farther input,
the structure-borne and air-borne direct propagation are separate because of the large propagation delay difference.
However,
we cannot directly identify the structure-borne and air-borne direct peak due to the random initial propagation delay.
To combat the ambiguity,
we perform a calibration scheme based on the property of adjacency with these two components.
To identify the structure-borne component,
we use an algorithm to select the peak points corresponding
to the structure-borne propagation and air-borne direct propagation
to select the exact peak points.
One major challenge
preventing precise gesture classification is the signal corruption.
The fundamental idea of this model
to decompose the acoustic data into two parts:
one part is rich of gesture patterns
but insensitive to the user while another
includes user-related information
but insensitive to the gesture.
We propose a deep adversarial learning algorithm in order to learn pure
gesture-specific representations :
which is composed of three crucial components:
signal decomposition, user-specific classifier, and gesture-specific classifier.
For evaluation,
we recruit 10 volunteers aged from 23 to 30 years old.
All volunteers are asked to perform 10 gestures as shown in Figure and repeat 100 times with index finger
These two figures illustrate the comparison
between our approach and other classifiers over 10 gestures and 2 situations.
The results show that: our approach obviously outperforms all the baselines,
including typical non-deep learning classifiers and competitive deep learning models,
over all gestures and all situations.
To have a closer observation,
the confusion matrix of the data set relative to the person
who has the average performance over two situations
are reported in the two figures.
To study the impact of parameters in terms of the classifier model
we examine the system performing by tuning three important parameters,
such as , batch size, learning rate and epoch.
The left figure shows the average recognition accuracy with different batch size and learning rate.
The right figure shows that the training loss degrades gradually with
with the increasing epochs and the degradation tends to converge after 200 epochs.
Thank you, that's all