Hi, I'm Blaine Lewis. And in this video I'll be presenting a
new interaction technique called KeyMap. KeyMap is a keyboard
shortcut visualization designed to help improve keyboard
shortcut memory by taking advantage of Norman's mapping.
This is research I performed at the University of Waterloo while
completing my master's degree, as well as the University of
Canterbury while doing a research assistantship. The idea
behind KeyMap is extremely simple, rather than showing
keyboard shortcuts hidden away in linear menus we instead,
display them directly on top of the keys of a virtual keyboard.
But before I get too much into KeyMap, I first want to talk
about why we should care about keyboard shortcuts in the first
place. And the biggest reason is that keyboard shortcuts are
fast. We don't have to home back and forth between the keyboard
and the mouse and they're not constrained by the steering or
Fitts law. The mouse stays in the work area. It doesn't bounce
back and forth between what you're working on and the
toolbar. You just keep working on what you're working on. But
the problem is that nobody uses them. Only 19% of Firefox users
Use Ctrl F to invoke the find command. But why don't people
use them? I just said they were fast. The reasons for that are
psychological biases and lack of design, the two psychological
biases that are important here are the productivity bias where
people are unwilling to sacrifice productivity now for
productivity in the future, and satisficing where people are
unwilling to or where people will have a threshold of good
enough when they make a decision. So they look at linear
menus and think, Oh, that's good enough, and they never even
consider keyboard shortcuts an option. But the two reasons I
really want to focus on in this talk, our keyboard shortcuts are
hard to remember, and they're hard to discover. They're hard
to remember because we don't design for it. When you look at
the information we give for remembering a keyboard shortcut
from a linear menu, we give the label like undo and the keyboard
shortcut like commands that we don't tell you where on the
keyboard it is, or what shortcuts are nearby, there's
really not much useful information to remember what
keyboard shortcut it actually is. But keyboard shortcuts are
also hard to discover both implicitly and explicitly.
Implicitly when they're displayed in the near menus.
We're scanning on the left side of the commands, but not looking
all the way over on the right to look at the keyboard shortcuts.
But they're also explicitly not very discoverable. For example,
on toolbars, you have to hover the buttons to see what a
keyboard shortcut is. And then sometimes they're even hidden
behind keyboard shortcuts themselves. So you have to press
Command slash to open the panel on the right here. There has
been some research into how we can approach some of these
problems. For example, Grossman et al. looked at how we can
change linear menus to help us remember shortcuts by using
audio cues or even delays. ExposeHK introduces the idea of
rehearsal where novices and experts perform keyboard
shortcuts the same but maybe novices but the visualization.
And hotkey palette looks at putting your windows on top of a
virtual keyboard, similar to how keymap puts command labels onto
both virtual keyboard. The idea we take advantage of in KeyMap
is called Norman's mapping. It is the idea that the controls
that we use should be aligned with the effect they have. So
for example, if we have a stove if the controls are aligned in a
line, but the burners are in a square that's not naturally
mapped, because it's unclear how I can map from the dial to the
burner, do the dials act in a clockwise manner, where the top
left is where we go around from the top left, or maybe clockwise
where we go counterclockwise where we go the other way. Maybe
we go in lines where we do the first round on the second row.
It's unclear without adding some extra visualization. However,
when you arrange the dials in a square that becomes immediately
obvious which dials control which burners that is In the
case of keyboard shortcuts right now, because there's no natural
mapping between how we enter a shortcut and how it's displayed
in the linear menus. So what does Norman's mapping look like
for keyboard shortcuts?
And the answer is key map.
Rather than showing keyboard shortcuts hidden away in linear
menus. Instead, we show them directly on top of keys on a
virtual keyboard, you can see that each command is displayed
on a key as well as the actual key letter behind that. And we
also took advantage of other design principles or design
ideas like displaying the nubs that you see the physical
numbers you see on your keyboard right now on F and J, and
displaying those on top of the virtual keyboard that we display
so that you have a physical landmark to help you remember
where keyboard shortcuts are. Key map works very simple. You
press and hold a keyboard shortcut modifier like command,
and then the keyboard shortcuts appear on the screen
after a short delay
then Complete your selection just by entering the rest of the
keyboard shortcut. If you want it to enter, find, then you
would just press F, and the command would be executed and
the menu would disappear. KeyMap is always displayed in the
bottom left. And this makes keyboard shortcuts a first class
citizen so you can always see that they're there. And now they
can become a default learning mechanism rather than kind of an
aside. For experts, there's no difference because the short
delay isn't even noticed you never see key map in the first
place because you can enter a shortcut facet and the delay
executes. And for very novices, what they can do is they can
actually click through the keyboard shortcuts, like they
can click on command and then click on F to enter find and
that helps them learn at least some knowledge and then they can
use that knowledge to learn keyboard shortcuts in the
future. There's two principles behind KeyMap. The first one is
the idea of cue target Association, which should help
memory and when we have to remember keyboard shortcut,
we're performing a cued recall process where we have a cue,
which is the visualization or display and a target, which is
the entering of the actual shortcut. And in linear menus,
the cue is the linear menu, and the target is the shortcut. But
there's no association here, they don't even look visually
similar. So this association is very weak for the current state
of the world. But for keymap, we can look at those two images and
it's immediately obvious that KeyMap has a strong association
with the way you're entering. The second key principle behind
KeyMap is the idea of spatial emory. With KeyMap you can tell
hat some shortcuts are line be ide each other semantically.
f I look at this linear menu her , I can't tell the cut, copy
and paste directly beside each o her on the keyboard. But ke
map makes it extremely obvious, and it makes sense that some th
ngs should be semantically and physically grouped toget
er. Other things like Find and Replace are
placed near each other.
We wanted to compare keymap to the state of the art and in this
case, we said that the state of the art was ExposeHK. I mentio
ed ExposeHK earlier. And the way it works is when a user presses
and holds a keyboard shortcu modifier. The linear menus a
l appear at once and then th user waits or looks for the pro
er shortcut to enter and finishe it by pressing the corresp
nding letter key. And that co pletes the selection. And now
a novice and a user or an expe t are doing the same action.
We had three hypotheses for our experiment, and we thought
that keymap users would remembe more shortcuts, they would i
cidentally learn more shortcu s, and they will also have fa
ter practice selection times t at expose AK. Our experim
nt consisted of two differe t tasks. The first task was a s
lection task where they practic to learn a set of keyboar
shortcuts, they can enter u ing guided mode where they ca
see the visualization and the can also enter it just using a
keyboard shortcut without the visualization. And they ca
also use the mouse base interac ion where they click through
the menu. There was also the rec ll task where users were given t
e same set of commands that th y'd already seen. But in this ca
e, they couldn't use any of the isualizations. And they just ha
to enter shortcuts.
and here we can see these are inter shortcuts, but there's not
much to see here because they don't get to see the
visualization. Our experiment design was between subjects we
compared expose HK to keymap and recruited 98 Mechanical Turk
crowd workers. They each were assigned 13 commands distributed
over a Zipfian distribution to make some commands more frequent
than others, just like in the real world for commands like u
do. Participants started by performing the selection task
where they first practice some commands and they did six blocks
with 25 selections each, and then they continued on to the
recall task where they entered all 13 commands and then also
find that they hadn't seen before to test incidental
learning. After 25 hours passed, we wanted to test their long
term memory and we invited the participants back to perform the
retention task where they did the same thing as the recall
task. We found that keymap improve short term recall there
is a significant difference between key mapping ExposeHK o
one command where key map u ers remember 10 and expose e
ght came from number nine. T is plot is a violin plot. But f
r discrete distributions, we c n see that on the right for h
gh frequency commands key map u ers and ExposeHK users both re
ember about the same number of commands. But as we start to go
towards low frequency co mands, this is where we start to
see a difference where KeyMap us rs are remembering more sh
rtcuts than ExposeHK users are We also found a significant dif
erence in long term ret ntion, which is what's rea
ly important because we want use s to remember session after ses
ion with the shortcuts are and we found that key map users rem
mbered 10 commands on me and exp se educate users remember onl
5.5. This is a really big dif erence of four and a half sho
tcuts out of only 13 pos ible. And this is really imp
rtant if we each knew four and a half more shortcuts, then tha
would be really impactful for our daily lives.
And we see the same patterns of differences in frequency.
We also found that key map likely creates incidental
learning across the entire experiment to the map users were
able to remember 19 commands that they'd never seen before.
Whereas exposed H key researchers were only able to
remember six and this is also a statistically significant
difference. We didn't find any evidence that keymap is faster
than expose agk. Both mechanisms gave us a selection times of
around 3.2 to 3.4 seconds. So to conclude our experiment, we
found that keymap users remember more shortcuts and explanation
users and they were also able to incidentally Learn more
shortcuts. However, we weren't able to find any evidence that
keymap users have faster practice selection times. As
part of this work, we also built the Chrome extension. This lets
you use it today. It's open source and available in the
Chrome Web Store. And you can download it and use it with
common applications like overlay. The cool thing about
key map is that it's a direct augmentation or replacement of
current command entry techniques. So it just sits in
the bottom left to unobtrusively and you can just use it. Here's
an example of me using it to write our paper and Overleaf, I
am editing the abstract and adding bold and italics to it.
And keymap appears for shortcuts there may be I don't know.
keymap works for two other applications, Dynalist and S
ack but it's also open source w ich means that if you want to a
d new command mappings to it, y u can do that. Thank you for w
tching this talk. Today I introduced key map which is
a new interaction technique th t takes advantage of Norman
s mapping to improve shortc t vocabulary by using a nov
l keyboard visualization. If y u want to learn more about t
e paper, or if you want to lea n more about the interacti
n technique, please read t e paper. And also check out t
e Chrome extension. And the Chro e extension link is in the pape
, but it's also available at y website blainelewis.ca/keyma
. I'd also like to thank my co authors, especially Greg who s
ares first co authorship with e
and thank you for watching this talk.