- At our September Virtual UX Conference,
an attendee asked Dr.Jakob Nielsen
what his thoughts were
on how the system usability scale,
SUS is used in the real world.
Here are his thoughts.
- So the system usability scale is interesting
because it's one of the first sort of systematic ways
of assessing user satisfaction
with this fixed set of questions.
And so, because of that,
one of the benefits of that
is that it's been used for a long time.
So that's a very large dataset of information
about how responses have been over actually decades, right?
So, that's one of the benefits of that scale.
However, that said,
I feel that it's being overused
and it's not because it's old
because there's nothing wrong about being old.
I mean, it's established that's good,
but some of the wording is a little clunky
but the main point that I think I have a gain to just really
that it's a lot of questions.
And what I want to focus on
in any user research is really behavior.
And if you wanna spend more of your very precious time
with any research participants,
they are hot to recruit
and hot particularly certain job categories of people,
very difficult to get your hands on.
And so whatever time you have with them is precious.
And you wanna spend that mostly
on observing them being users, doing things,
seeing what comes easy, what's difficult,
how they perform their different tasks
and not so much about asking them
in detail about their opinion
because as we all know,
opinion is the weakest form of user data.
And that's not as good as driving forward
to design as observation.
So you wanna be the expert no user researcher.
Observing people's behavior and deducing
from that what should be done.
You don't wanna ask them,
you wanna find out by observing them.
People are experts in being users you know,
but their opinion is less important.
That said, we do want people to opinion
because one of our quality criteria is satisfaction.
And we do want people to like our designs
and maybe in the future,
we can assess some of these engagement type of issues
through some type of face recognition
or sort of artificial intelligence.
They can find out how happy people are by looking at them,
by doing various types of biometrics like skin conductivity
and so forth.
But the truth is that these methods right now are very weak
and often quite misleading.
And so right now, the best way we have
of knowing whether people like something
or not is just to ask them, right?
How much do you like?
In one to seven scale or something like that
and tell me how much you like it.
But I feel that the,
just ask that one question you know,
just ask one satisfaction question.
I don't think we should need to ask 10 of them
and get a very elaborate set of profiles
about different ways in which people like it
because that's not really
what's important.
I mean, even for things like let's say the visual design
which is something that you often wanna say like,
is it appealing or not?
But often you can actually get a better assessment of that
by noting what people comment on essay using the design.
Are they commenting on this looks cool
or this looks boring?
Those are much, those are in context.
And the conscious of views,
comments are often much more valid, more interesting
and also again because they give you a feeling
of why people say something.
Also better at driving design change
which again is why we're doing the research.
So, bottom line I mean,
the benefit of SUS is that it has a lot of data
in it already because it's been used for decades.
The downside, I think is it consumes too much time
and attention and we should pay less attention to opinion
and more attention to observation.
And I would say,
just ask people one question for satisfaction.