- Thanks for the introduction.
So touch screen watches are more and more common,
but if you look at the standard interaction techniques
on these devices, you see a lot of this.
You have course-directional swipes through screens
with only one or just a few buttons.
I counted, and to select an album on the Android Music app
required four swipes and two taps
just to get to the list of items
and then you'd have to do more interactions from there.
This is really slow, and it defeats
the point of smartwatches
which is to have these really fast interactions
while you're doing other things.
I mean, these devices, they're competing with
taking a smartphone out of your pocket
in a lot of the cases
so interactions have to be really really fast.
So in this work we're interested in exploring
higher bandwidth input for touch-screen watches
and specifically we're interested in the question,
can spatial menus enable rapid command selection
on touch-screen watches?
For those of you who aren't familiar,
in a spacial menu, commands are associated
with spacial locations and then executed
by indicating a spacial location to the system.
Here is an example of the spacial menu
for tablets called FastTap.
The menu has a novice mode, where you touch
a menu button with one finger to open the menu,
and then use another finger to select the item.
But then, over time you build up a spacial memory
for where the items are, and then you can
rapidly select items with a two-finger tap
and you'll see that in a second here.
So you see some selections there that are more rapid.
These menus are like keyboard shortcuts for touch screens
and they're extremely fast.
We wanted to see,
could we apply this approach to touch-screen watches?
But it really wasn't obvious if this would work
because these devices have such a small input space.
So overall, there's just not a lot of area to interact with
and also, multi-touch becomes more challenging
because you have occlusion
and fingers being super close together.
I'll kind of just spoil the talk right now
and say that we found that spacial menus
can be adapted to work on tiny touch screens.
They enable fast selections
about one second per selection with practice.
They can support up to 18 commands at that speed shown
using a design that I'll show.
They can be learned quickly
and they have relatively low error rates.
But let me just step back for a second.
To get to these findings,
we looked at two execution mechanisms
which I'll show you in a second.
We investigated the impact of button size
on performance in accuracy,
we looked at this idea of overloading locations
to accommodate additional commands that I'll describe,
and then we evaluated our techniques against the status quo.
I'll just kind of go through these in turn.
The first execution mechanism we looked at
was just a straightforward application
of the FastTap menu that I showed earlier to a watch screen.
You have this kind of novice mode where one finger
opens the menu and then the other selects an item.
But then, once you've built up some spacial memory,
you can do an expert mode
where you just tap the two simultaneously.
The second technique, TwoTap, is similar,
but it uses two sequential taps to achieve the same thing.
So in a novice mode you tap visual search
and then tap a target item.
Here's one more time.
But then once you know the item locations,
you can do two taps quickly without a visual search.
And one of the hypotheses that we had in this work
was that over time these two actions would
become integrated into a single kind of
chunked motor action similar to how when you
unlock a lock screen on your phone
you don't think about hitting
each of the individual buttons,
you just kind of perform the whole thing.
So next, we conducted a study to understand
basically how many locations we could fit
on one of these tiny screens.
We looked at three grid sizes, shown here.
And for each grid size,
we had participants perform sets of trials
simulating the two WristTap interaction techniques.
See, here's what that looked like.
This is simulating the fast selection.
Again, the fast selection on a 4x4 grid.
And then this is the novice selection on a 3x3.
We had 14 participants, and each of them performed
trials for each grid size and selection method.
In terms of performance, we found
a significant main effect of both the grid size
and the selection method.
The two-step novice selections
took between 1.2 and 1.4 seconds,
And the one-step expert selections were between
600 and 800 milliseconds.
But the key thing here is error rate.
For error rate both the 2x2 and the 3x3 grids
had a median of zero errors
and both had averages that were under 1%.
So that's quite good.
But then the rate of errors was higher for the 4x4 grid.
So based on these results, we decided to go with
the 3x3 grid because it has a near zero error rate
and comparable performance to the 2x2,
but it can fit more items.
But with a 3x3 grid, that would mean
we could only support eight commands.
You've got nine spots and you need one for the menu button.
So to enable more commands we investigated
the idea of overloading grid locations.
To accomplish this, we have multiple menu buttons,
each with their own set of commands.
You have the three menu buttons along the bottom there.
But this kind of raises this question,
how does the overloading of menus
impact people's ability to adopt
this expert selection technique?
Remember, these techniques really only get fast
once people build up the spacial memory
and can perform them without the visual search.
So we want to make sure that this could still be done
when we've got these three overlapping menus.
To test this, we had participants perform trials
where they were asked to select prompted items.
There's a one-step selection.
This is what it looked like.
We selected a subset of the items
in these three menus so that for some grid locations
participants only needed to learn one item,
but for others they had to learn two overlapping ones
or in one position three overlapping items.
The study had three training stages
where we encouraged participants to try
and learn the item locations followed by
a testing phase where we asked them
to just use the expert technique exclusively
even if they weren't fully confident in using it yet.
Here's the rate of use
of the fast expert selection technique over time.
We can see that in less than about 20 minutes,
that was the duration of the study,
participants went up to about 80% usage
of the expert technique.
You can see in the testing phase it was near 100%,
but that was because we explicitly asked them
to use it there.
For that final testing stage, in terms of performance,
the average trial time when there's no overlap,
where you're only learning one item at a location
was faster, but not significantly so.
In terms of accuracy, overall we saw
a median error rate of 3.6% for users.
We really observed a high variation
in error rates across users.
About half of the users had error rates of less than 2%,
many of them with zero.
The other half were quite a bit higher,
in the 4% to 10% range.
We also looked at it breaking down by the number
of overlaps and we found greater error rates
with more overlapping items.
Overall, at the end of the day,
what this suggests is that overloading
does slow down learning somewhat,
but users can learn overloaded menus
and use them quickly and accurately.
And in 20 minutes they're able to learn half
of the 18 items in the three menus.
Finally, we tested our techniques against existing methods.
For this study, we had participants perform
the trials while standing.
The watch would buzz, and then they would raise their arm,
do a short block of trials
and then they'd lower their arm again
and wait for the next block.
We had a video playing of a walking tour of Amsterdam
and that was just to simulate
some distractions in the environment.
We had three conditions, here's the WristTap condition,
here we didn't have the swiping through targets
you just had to select them to complete it.
Here's the TwoTap condition,
you've seen these before.
Finally, we had the Swipe-and-Tap condition.
This was designed to simulate the interaction techniques
that we see in many of the current smartwatch apps.
You have something where you open a menu,
and then you have this vertical list
of things you have to swipe through and then select from.
Each of our participants experienced
all three of the menu systems.
This was a within participants design.
For each menu system there was a training phase
and then a testing phase where we were gathering data.
So what did we find?
Here is the error-free trail completion times
for the testing stage by technique.
We see that the WristTap selections took
just under a second to perform.
The TwoTap took about a second.
Swipe-and-Tap was more than double the time
at 2.6 seconds per selection.
A repeated measure found that WristTap and TwoTap
were each faster than Swipe-and-Tap,
but there was no significant difference
between the two techniques.
In terms of error rates, we did an analysis
for both selection errors where and incorrect item
was selected but also for category errors
where the participant opened the wrong menu
but then they could still recover from them.
We could see that WristTap and TwoTap both have higher
error rates than the control condition, Swipe-and-Tap.
But these errors are still pretty low, around 3% or lower.
WristTap had more selection errors which makes sense
because the fast selections in WristTap
were performed in one action, so there's no way
to recover once you've started the action.
TwoTap on the other hand, had more category errors
but fewer selection errors,
and that actually conversely suggests that with TwoTap
people can recover once they open up the incorrect menu.
So both of our techniques were faster than the status quo,
but they kind of each have distinct characteristics
that are worth highlighting.
First, WristTap's expert mode.
Because it uses multi-touch to do selections,
it can kind of co-exist pretty well
with the interaction methods that we
commonly already see on watches.
It could be applied in existing app as
kind of a shortcut method without too much modification.
Second, WristTap had the disadvantage
of higher selection errors, which would suggest
that it should be used in scenarios
where an error has a low impact
or could be quickly reversed.
I kind of think of it as you could imagine
a quick gesture to put my phone into airplane mode
or something like that.
If there's an error there, it doesn't really matter,
but it would be a very fast interaction.
And finally, it was slightly faster than TwoTap.
Though like I said, in our study
we did not find a significant difference there.
TwoTap on the other hand is conceptually simpler.
It's only one selection method,
you don't have this difference between open, touch,
and two fingers together.
It also has the lower selection error rate
because these category errors are recoverable.
It also could conceivably support more menu items
than WristTap because you don't have to have a dedicated
grid location for the menu button.
You could have grid locations for both menu buttons
and an items one once they're tapped.
There's more information on that in the paper,
in case you're interested.
So I'll just end with some takeaways.
Spacial menus can work on tiny touchscreens.
Overloading three 3x3 menus we found to have
a minimal impact on adopting the expert selection technique.
They're significantly faster than standard techniques
with low error rates.
They have different comparative pros and cons,
which I've discussed briefly.
With that, I'll finish, and I'd be happy to take questions.
Thank you.
[applause]
- [Host] So we've got five minutes for questions.
There's a microphone over there
if you would like to ask Ben something about the techniques.
- [Man] Did you consider using or testing gestures
for swiping as the ones you see
in some novel keyboards for smart phones?
- Like swiping between different grid locations
to do something?
We didn't look at that, but that is kind of interesting.
I have seen some text-entry methods that
even for watches that use that kind of swiping approach,
but we didn't look at it for doing command selection
not in this were.
- [Woman] Can I ask just...
- Please, yeah.
- [Woman] Why do you think that the TwoTap
had the category errors?
- It's a good question.
I'm actually not quite sure on that.
I'm not sure why it had a higher error rate
for the category errors.
I was going to say it could be just the difficulty
of targeting the items in the 3x3 grid.
But our first study we looked at that
and the error rate was like less than 1%.
It's a very good question, and yeah,
I'll think about that I guess.
- [Woman] Do you think that this is the work
from users who are mobile?
- Yeah, well, I'm not sure.
We're really kind of interested in,
for the first two studies they were performed sitting down.
And so in this third study we were like,
okay, we have to doe something that is
a little bit more like you're on the go,
you're doing something so we did this standing up.
We didn't go as far as to get a bike or something like that,
but it is really a good question.
One thing that we did do in the paper
that I didn't talk about for time
is we had a block at the end of the third study
where we blanked out the entire screen and said,
now do the selections.
You saw the stimulus, but you didn't see
the menu items at all.
For both of our techniques, people could get 80% accuracy
with absolutely no visual feedback whatsoever.
We didn't test it for the control condition
because that just would not work 'cause that's just
relies so much on visual feedback.
That might boat a little bit well
for doing it while riding a bike
but there's still shaking and other stuff there.
It's a good question.
- [Host] Okay, any other questions?
Okay, thanks very much, Ben.
- Thank you.
[Applause]