Ah, well – it's a lovely day here in Tiree.
I'm looking out the window again.
But how do we know it's a lovely day?
Well, I could
– I won't turn the camera around to show you, because I'll probably never get it pointing back again.
But I can tell you the Sun's shining.
It's a blue sky.
I could go and measure the temperature. It's probably not that warm,
because it's not early in the year.
But there's a number of metrics or measures I could use.
Or perhaps I should go out and talk to people
and see if there's people sitting out and saying how lovely it is
or if they're all huddled inside.
Now, for me, this sunny day seems like a good day.
But last week, it was the Tiree Wave Classic.
And there were people windsurfing.
The best day for them was not a sunny day.
It was actually quite a dull day, quite a cold day.
But it was the day with the best wind.
They didn't care about the Sun; they cared about the wind.
So, if I'd asked them, I might have gotten a very different answer
than if I'd asked a different visitor to the island
or if you'd asked me about it.
And it can be almost a conflict between people within HCI.
It's between those who are more *quantitative*.
So, when I was talking about the sunny day, I could go and measure the temperature.
I could measure the wind speed if I was a surfer
– a whole lot of *numbers* about it –
as opposed to those who want to take a more *qualitative* approach.
So, instead of measuring the temperature,
those are the people who'd want to talk to people to find out
more about what *it means* to be a good day.
And we could do the same for an interface. I can look at a phone and say,
"Okay, how long did it take me to make a phone call?"
Or I could ask somebody whether they're happy with it:
What does the phone make them feel about?
– different kinds of questions to ask.
Also, you might ask those questions
– and you can ask this in both a qualitative and quantitative way – in a sealed setting.
You might take somebody into a room, give them perhaps a new interface to play with.
You might – so, take the computer, give them a set of tasks to do
and see how long they take to do it.
Or what you might do is go out and watch
people in their real lives using some piece of
– it might be existing software; it might be new software,
or just actually observing how they do things.
There's a bit of overlap here – I should have mentioned at the beginning –
between *evaluation techniques* and *empirical studies*.
And you might do empirical studies very, very early on.
And they share a lot of features with evaluation.
They're much more likely to be wild studies. And there are advantages to each.
In a laboratory situation, when you've brought people in,
you can control what they're doing, you can guide them in particular ways.
However, that tends to make it
both more – shall we say – *robust* that you know what's going on
but less about the real situation.
In the real world, it's what people often call "ecologically valid"
– it's about what they *really* are up to.
But it is much less controlled, harder to measure – all sorts of things.
Very often
– I mean, it's rare or it's rarer
to find more quantitative in-the-wild studies, but you can find both.
You can both go out and perhaps do a measure of people outside.
You might – you know – well, go out on a sunny day and see how many people are smiling.
Count the number of smiling people each day and use that as your measure
– a very quantitative measure that's in the wild.
More often, you might in the wild just go and ask people.
It's a more qualitative thing.
Similarly, in the lab, you might do a quantitative thing – some sort of measurement –
or you might ask something more qualitative – more open-ended.
Particularly quantitative and qualitative methods,
which are often seen as very, very different,
and people will tend to focus on one *or* the other.
*Personally*, I find that they fit together.
*Quantitative* methods tend to tell me
whether something happens and how common it is to happen,
whether it's something I actually expect to see in practice commonly.
*Qualitative* methods – the ones which are more about asking people open-ended questions –
either to both tell me *new* things that I didn't think about before,
but also give me the *why* answers
if I'm trying to understand *why* it is I'm seeing a phenomenon.
So, the quantitative things – the measurements – say,
"Yeah, there's something happening. People are finding this feature difficult."
The qualitative thing helps me understand what it is about it that's difficult and helps me to solve it.
So, I find they give you *complementary things*
– they work together.
The other thing you have to think about when choosing methods
is about *what's appropriate for the particular situation*.
And these things don't always work.
Sometimes, you can't do an in-the-wild experiment.
If it's about, for instance, systems for people in outer space,
you're going to have to do it in a laboratory. You're not going to go up there
and experiment while people are flying around the planet.
So, sometimes you can't do one thing or the other. It doesn't make sense.
Similarly, with users – if you're designing something for
chief executives of Fortune 100 companies,
you're not going to get 20 of them in a room and do a user study with them.
That's not practical. So, you have to understand what's practical, what's reasonable
and choose your methods accordingly.