Hi everyone, my name is Katherine and in this video, we will demonstrate how hotkeys
can be capitalised to unify command selection mechanisms on both phones and tablets.
First of all, command selection mechanisms vary from one operating system to another.
For instance, imagine you are browsing on an iPhone, and you want to find a particular word,
do you know how to do so?
First, we have to tap on this share icon and then slide up to reveal more items,
before finally tapping on the ‘find’ command.
Well, how about on Android?
Instead of the share icon, we have to tap on this kebab menu icon,
and then tap on the ‘find’ command.
So, there are 2 main problems we want to highlight.
First is how slow and difficult command selections are on our mobile devices.
A frequent task like finding words should not be this tedious and slow.
Another example is the undo command.
Most iPhone users do not know they can undo by shaking the phone,
unless they google it out or trigger it by accident.
The bigger problem we have to acknowledge is that,
there is a lack of consistency across applications and OS.
For the same find and undo command,
users have to look for different menu icons or invoke the shaking gesture.
This is a problem because, especially in today’s context,
where we often interact with multiple devices,
inconsistent mechanisms force users to
constantly adjust their techniques as they switch between devices.
Hotkeys, on the other hand, are well-established and unified.
So these got us thinking,
what if we could leverage our prior knowledge of hotkeys
from physical keyboards, and transfer it to multitouch keyboards?
We call this approach SoftCuts,
which is an abbreviation for Soft Keyboard Shortcuts.
Attempts to implement SoftCuts can be found on commercial devices
like Samsung, Microsoft Surface tablets and on applications like Swype.
While our previous work had investigated which input method is suitable,
there are many remaining questions on how to realise the potential of SoftCuts,
which we would like to focus on in this work.
First, we compared the usability of three visual representations
through semi-structured interviews.
First, we have letter only.
Second, we have both letter and name of command.
Third, we have both icon and name of command.
This is a demonstration of the Notes app prototype equipped with all three designs.
Results show that there was a preference for Icon+Name,
and Letter+Name could serve as the next best alternative.
Moving on to our second experiment, we studied how discoverability of SoftCuts
is affected by familiarity and saliency of modifier keys.
We used the same Notes app prototype
but varied the type of modifier keys used,
and also varied the saliency of the modifier keys
between keyboard and keyboardless conditions.
The only difference between keyboard and keyboardless conditions
is in the initial presentation of the prototype.
Here we demonstrate when a custom modifier key is used in both conditions.
Results show that discovery rates were high regardless of conditions
and familiarity to hotkey knowledge/usage helps participants to discover SoftCuts.
Next, in our third experiment, we studied how easily users learn command mappings.
The abstract set comprises 16 countries
while the realistic set comprises 16 existing hotkeys
from Microsoft Word application.
Results show very small difference in performance between the two mappings.
Participants were only 5% slower in abstract mapping,
but overall 1% more accurate than realistic mapping.
The interesting result to note is in terms of retention rate.
We observed that many of their incorrect answers are adjacent to the target key.
If we combine the correct key with the adjacent keys,
we can see from the red dotted line
that the rate for abstract mapping increases by 25%.
This suggests that while retention for new mapping is not perfect,
participants could still estimate the general area,
where the item was located even after one week.
And this is good considering that
realistic mapping benefits from years of exposure and practice,
while abstract mapping was only used for 15 minutes.
In our final experiment, we studied the performance difference between grid and keyboard layout.
The grid layout is to mimic that used in FastTap.
This is a demonstration while using a phone,
and this is a demonstration while using a tablet.
Results show that while there was no significant difference in terms of accuracy,
the grid layout can be up to 220 ms faster than keyboard layout.
Despite that, it is worth noting that
the keyboard layout can save 60% of space
and offered more spatial stability,
which is critical for efficient performance.
Thus, the grid layout is overall at a disadvantage
because it does not benefit from any standardised mapping,
unlike that by hotkeys for keyboard layout.
Overall, the results from our four studies show that
SoftCuts can reliably be used as a unified command selection mechanism.
Specifically because of the following reasons.
Firstly, SoftCuts is not only fast and accurate on both phones and tablets,
but also requires minimal training for maximum performance.
Secondly, the design with icons are helpful for users who are new to hotkeys mapping,
while the design with letters can be considered for experts who desire customisation.
Next, we recommend using familiar labels depending on the OS:
Command key for iPhones/iPads and
Control key for Android/Windows devices.
Last but not least, we showed that users not only can learn multiple mappings,
but also take advantage of their prior knowledge.
All of our four studies have only been utilising one modifier key.
It is a feasible direction for future work to expand the input vocabulary of SoftCuts,
by exploring the effects of nesting additional modifier keys like option or function.
Another future work is to standardise hotkey mappings comprehensively.
While basic commands like cut, copy, and paste
have been implemented without discrepancies,
there is room for improvement for other more advanced commands.
For more detailed descriptions and discussions,
please refer to our full paper. Thank you.