- When I'm presenting our measuring UX and ROI seminar,
I sometimes get questions about the differences
between quantitative and qualitative research.
In UX, we often use qualitative research
to gather insights or observations
about users and products or services.
Popular qualitative UX research methodologies
include interviews, qualitative usability testing
and field studies.
With qualitative research our goal is often
to discover problems or opportunities
in the user experience,
to investigate why something is happening
or to find ways to fix a problem.
For example, maybe I conduct a round of interviews
to understand why a feature isn't as popular
as I expect it to be.
Maybe through the course of my interviews
I learn that our users don't understand
the point of this feature or how they're supposed to use it.
I could use that information with my design team
to help us decide whether this feature should be phased out
or how we should redesign it.
Because our goal is to identify problems
and find ways to fix them,
it's okay to test with a small number of users.
Our rule of thumb for qualitative research
is five to eight users per user group.
To explain why, we can think about a pothole in the street.
If you watch five cars drive down the street
and each one keeps hitting this pothole,
you know it's a problem.
You don't need to watch 45 more people
drive into that pothole before you decide
that it should be fixed.
When we're conducting quantitative UX research,
instead of gathering insights,
we're gathering numbers that describe
some aspect of the user experience.
We refer to those numbers as UX metrics.
Popular quantitative UX methodologies
include analytics, A/B testing, surveys
and quantitative usability testing.
You might wanna use a quantitative methodology
if you're trying to determine
the priority or scale of a problem.
What proportion of my users
are impacted by this particular problem for example?
You might also use a quantitative study,
particularly A/B testing,
if you wanna compare alternative design options
and use data to inform your decision.
Finally, quantitative research is also great
for benchmarking the UX of a product or a service.
Essentially that means tracking
your user experience over time
to make sure that you're improving
in the ways that you expect.
You can also use that data
to calculate return on investment.
Often with quantitative studies
our goal is to compare things.
Maybe we're comparing our current version of our product
against our older version.
Or our product to our competitor's product.
Or maybe we're comparing two different alternative designs
for a specific layout.
A major benefit of quantitative research
is that it gives us the ability
to determine whether the difference
between two numbers is statistically significant.
In other words, to know that that difference
is not likely due to random chance.
That's part of the reason
that we need larger sample sizes for quantitative studies.
If we don't collect the right amount of data,
you may not have a reliable result.
There are different recommended minimum sample sizes
for each quantitative methodology
so it's important to check that minimum
whenever you are beginning to plan a quantitative study.
Qualitative and quantitative research
give us different kinds of information
about the user experience.
The best strategy is to use a variety of methods
in your UX practice.