This video is about word suggestions,
and how they benefit people in function of their accuracy, and the device being used.
My name is Quentin Roy. And this work is a collaboration with Sébastien Berlioux,
Géry Casiez, and Daniel Vogel.
Word suggestions is a technique to speed up text input by predicting what users are
typing before they do, and suggest shortcuts, so they do not have to type the entirety of a word.
Word suggestions are widespread they are available on virtually every mobile devices
but also on laptop computers. Million of users are daily exposed to word suggestions.
Still, we actually know quite little about how they are used, and how they can benefit users.
Previous work even showed they can in fact slow down users.
But where do we move from here? Does this means suggestions are not accurate enough?
How accurate do they need to be? Does it depend on how fast users can type to begin with,
for example the device they use?
In a nutshell, what is the impact of accuracy
and typing on the usage and benefits to entry speed of word suggestions?
To answer this, we designed a simple transcription task
using well established phrase set from Mackenzie and Soukoreff.
We included two factors, the accuracy of word suggestions,
ranging from upsettingly inaccurate to uncannily accurate, and the device used to type.
The device used to type was used as a proxy for typing efficiency.
Here is how the different conditions looked like.
Note that on phone, participants had to use only one hand.
For the accuracy factor, however, we had a problem.
Intuitively we have some idea of what it means for word suggestions to be
more or less accurate, but can we formalize it and operationalize it?
This is actually the second main contribution of our work.
I will not go into details, but our formalization of accuracy is based on maximum keystroke saving,
which measures the proportion of keystrokes that were saved thanks to word suggestions.
As an example, 0.5 accuracy means half of the characters are correctly suggested,
and thus can be saved.
But we also needed to control the position of the correct suggestion out of the three that
are shown to participants. To do this we built a bayesian model using data gathered from Mac OS.
Here is a some of our results. The coloured line indicate the device being used
On x is accuracy.
On y is how much keystroke were saved, which is of course bounded by the accuracy.
Keystroke saving increases with accuracy, though note that this is not linear. In fact,
participants did not use much suggestions, except when they were extremely accurate.
90% accuracy feels like the system is reading your mind.
But even then, they were virtually not used on laptop.
We were surprised to find no difference between tablet and phone. It is not shown here,
but participants were actually as fast when typing with two hands on a large tablet touch keyboard,
than on a phone with only one hand. We think it is the reason.
To conclude, word suggestions must be uncannily accurate to improve entry speed.
How beneficial they are depends on how fast users can type.
If we want to improve entry speed, designing faster typing techniques appears more effective
than improving the accuracy of word suggestions.