In this video, I’m going to talk about our work on the Rotational Dual Gaussian model.
This work is a collaboration between the HCI lab at Stony Brook University and Dr. Shumin Zhai from Google.
Modeling touch point distribution is important
because touch point distribution model can estimate the error of target selection, which
is important for interface design and soft keyboard decoder. The current touch point
distribution model is the Dual Gaussian model, it assumes that touch points follow a bivariate
Gaussian distribution centered at the target center, and the variance is quadratically
related to target size in both horizontal and vertical directions. The main issue is
that it assumes x and y are independent. It deviates from our observations. For example,
when the movement direction is 45-degree, the observed touch point distribution has
an orientation of roughly 45-degree We are going to take into account the finger movement
direction in modeling touch point distribution. We propose the Rotational Dual Gaussian model
to solve this problem. The finger movement direction is theta in this figure. Based on
the original Dual Gaussian model, we further assume that the major axis of the prediction
ellipse of the bivariate Gaussian distribution points to the finger movement direction, and
the other axis points to the direction perpendicular to the finger movement direction. We performed
eigendecomposition to calculate the covariance matrix of the RDG model. For details, please
refer to our paper. We also defined the width and height in the covariance matrix.
We propose three options. The projected width and height are the length of the line segment
formed by projecting the target size to the finger movement direction and the direction
perpendicular to the finger movement direction. The apparent width and height are the length
of the line segment intersecting the target along and perpendicular to the finger movement
direction and crossing the target center. For nominal width and height, if the movement
direction falls within the grey area, 𝑥-length is the nominal width and 𝑦-length is the
nominal height; if the direction is within the white area, it is the opposite. We empirically
chose the projected width and height for the Rotational Dual Gaussian model.
We used Ko et al’s dataset to empirically estimate the parameters of the four models
and evaluated them. The dataset contains 63 different target sizes and 16 pointing directions.
The WAIC shows that the RDG model with projected widths and heights has the strongest predictive
power. We also calculated the observed and predicted error rates. The root mean square
error of error rate shows the RDG model with projected widths and heights performed the
best. We intuitively compared the predictions of
touch point distribution by the two models. The blue dots are scatterplots of touch points
and the curves are the observed 95% density contours. The green curves are the 95% density
contours predicted by the original Dual Gaussian model, and the orange curves are predicted
by the RDG model with projected widths and heights. The orange curves resembled the observed
distributions and successfully predicted the orientation of touch point distribution.
We also evaluated the two models examining their predictions on x-y correlations. The
blue dots are the observed correlations. The grey curves are predicted by the original
Dual Gaussian model, and the orange curves are predicted by the RDG model with projected
widths and heights. The Rotational Dual Gaussian model successfully predicts the effects of
movement angle on x-y correlations while the original Dual Gaussian model assumes rho to
be 0. We tested the RDG model on smartwatch-based
typing tasks. We recruited 14 participants and collected over 29000 labeled touch points.
We implemented a statistical decoder and replaced the likelihood function with both models.
The RDG model improves the typing accuracy of 11 out of 14 users and keeps 2 users unchanged
compared to the original Dual Gaussian model. We also evaluated the RDG model in smartphone-based
typing tasks. We used Azenkot and Zhai’s dataset and performed a similar evaluation
as in smartwatch. The RDG model improves the accuracy of 6 out of 10 users and keeps 4
users unchanged. The RDG model has several applications. The
RDG model works for 2D rectangular targets, which are commonly found on current touchscreen
interfaces such as buttons, icons, hyperlinks, and menu items. It can also be used as the
likelihood function in keyboard decoders. Since it can accurately predict touch accuracy
for 2D pointing tasks, it can be used by UI designers in designing and evaluating touchscreen
user interfaces. In conclusion, we proposed the Rotational
Dual Gaussian model that predicts the distribution of touch point and advances the original Dual
Gaussian model by taking into account the finger movement direction theta. Second, we
defined the directional and amplitude constraints to be projected widths and heights. The model
we proposed more accurately predicts the error rate in target selection and improved the
decoding of soft keyboard.