Hello everyone, this is Xuhai Xu. The paper I am going to present is
Recognizing Unintentional Touch on Interactive Tabletop
Multi-touch tabletops are designed to combine the benefits of a physical table and a digital
computer.
However, most tabletops, especially those with a capacitive display, cannot descriminate
between touch events that are generated by different objects. Any part of the human body
or metal can easily generate touch events on the tabletop.
This makes users overly cautious during interaction to avoid triggering unwanted touches, driving
users away from a relaxed, natural experience.
Several works have been done to recognize unintentional input on small-scale touchscreen
devices. For example, Matero and Colley studied accident
touches on smartphones during daily usage. Their model achieves a true positive rate
of 79% and a false positive rate of 0.8%. Schwarz et al. looked into the same issue
on tablets and their random forest model achieves close to 98% precision and very small errors
per stroke.
However, tabletops can accommodate distinct interaction activities. They are more prone
to unintentional touches due to their increased surface area and a wider range of possible
inputs. For instance, users can easily lean on the tabletop or put an arm on the surface
to support themselves. These actions would be far less common when interacting with smartphones
or tablets.
In this research, we leveraged gaze, head and touch data to identify and filter unintentional
touches.
Our intuition comes from humans’ eye-hand coordination – usually where we look is
where we have interests, therefore where we touch.
We aim to answer three research questions in this paper:
First, we would like to have users to use a digital tabletop as if an ordinary physical
table. Currently, the behavior patterns on these two types of tabletops are very different,
but what’s the specific differences?
Second, can we train a model using gaze, head and touch data to recognize and filter unintentional
touches?
Third, once we have such a real-time model and a new interactive tabletop, will users
like it? what’s its usability?
To answer the first research question, we conducted a user study to compare the current
behavior pattern on a digital tabletop and a normal physical tabletop.
Meanwhile, the data collected from this study can also be used to train machine learning
models for later usage.
We used a two-factor within-subject design. The two independent variables are the type
of the tabletop and the user posture, with a Latin Square design.
12 Users were asked to complete two common tasks on tabletops: map navigation and photo
categorization.
During the study, all of their behavior data, including touch, gaze and head orientation,
were captured through multiple tracking systems. And After the study, two researchers went
through all touch data and annotate whether it is intentional or not according to the
behavior data and video recordings of the study.
We briefly summarize our findings of the behavior differences and their implications on feature
extraction here.
First, The majority of the gaze is in line with intentional touches, but not with unintentional
ones. This reveals the importance of the Distance between touch points and gaze points
Second, A large number of unintentional touches are closer to the edge compared to intentional
ones. This means side distance may be useful.
Moreover, Intentional touches usually involve just one to three fingers while unintentional
ones often appear in groups. The clustering of touch points can be an interesting factor.
In addition, Historical behavior affects current touches. For example, sometimes gaze leads
intentional touches shortly, the unintentional touches clustering area often spawn more unintentional
touches subsequently.
Based on these observations, we extracted four types of spatio-temporal behavior features.
Using these features, we compared a few machine learning models. The results show that the
Gradient boosting appears to be the best model.
We implemented and compared other state-of-the-art baselines. Our best model significantly outperforms
the baselines.
To answer the final research question regarding the viability of the algorithm, we implemented
a real-time filtering system by integrating the best gradient boosting model with the
tabletop.
Then, we invited another 12 users for the usability study. All users went through two
versions of the tabletop, one with our algorithm and one without our algorithm, with balanced
order.
They went through one map navigation task and one photo categorization task and then
report their subjective feelings
Our model performs well on the new set of users, with an F1 score over 90%. According
to participants' ratings, our new tabletop can successfully recognize their interaction
intention, and is more intuitive and more energy-saving.
To summarize, in this research, we propose to leverage gaze direction, head orientation
and screen contact data to identify and filter out unintentional touches.
We investigated user behavior pattern differences when interacting with digital tabletops versus
physical tabletops.
We then used four spatio-temporal feature types to train machine learning models to
identify touch intention. The Gradient Boosting model was able to classify unintentional touches
with an accuracy over 90 percent.
We then implemented a real-time filtering system. Our user study showed that our tabletop
could provide a more natural experience.
By linking users' gaze and head behavior to their touch behavior, our work sheds light
on the possibility of future tabletop technology to improve the understanding of users’ input
intention.
With this, I would like to end my talk. Thank you all for your attention! Hope everyone
is safe and healthy around the world. Happy to answer any questions!