- I'll start with a small exercise.
I'm going to show you some claims about our field,
and you have to raise your hand, if you think they are true,
at least partially,
so please raise your hand if you think they are true.
Our field should be more scientific.
Please raise your hand,
and keep your hand up, okay,
so let's go to the next claim.
Novelty is important in our work.
Great, good.
A good paper reports implications to practitioners,
and finally, cognitive modeling is passe.
Great, now thanks.
Kasper, what are you doing here?
This talk is about belief systems.
These are beliefs, like the ones that you just saw,
and this kind of belief systems are not like metta,
theoretical claims about our field,
but they are collections of beliefs
that can be not well articulated.
They are not necessarily internally consistent,
and they don't need to be value-free,
so beliefs like the ones that you heard before.
These are tremendously important
because they guide our thinking.
When you answer a question like what should I research,
or is this paper good, or are we progressing on a topic?
We are driven by these belief systems.
These belief systems, they become embodied in our practices,
and they become materialized.
We see them in the PCS,
and they are guiding our goal setting,
so that's why it's really important to try to study
and explicate what these belief systems are,
because if we have an ill suited believe system,
it can easily corrupt the whole field.
Now, do we have any reason to be concerned?
We see really excellent work here at the conference,
but if you take a step back,
and look at the field as a whole,
here's what Olson & Olson said 16 years ago
in a review paper,
that HCI's fragmented across topics, and theories,
and methods, and people.
We're also noticing that there's an increasing number
of disciplines contributing to our field,
and this means that it's harder and harder to say
what is the unique role that HCI has,
that doesn't reduce back to those other disciplines.
Finally, two years ago,
there was this argument about the big hole in HCI.
In a nutshell, what it means that the results
that we obtain in HCI
don't drive further research in our field,
so this is serious acquisition,
so we should be discussing what to do with these problems.
This paper is about one such attempt,
based on the problem-solving capacity,
or this concept coming from Larry Laudan,
and I can summarize it like this.
Instead of asking whether research is valid,
or follows the right approach,
it urges us to ask how its solutions
advance our capacity to solve important problems
in human use of computers.
We can expound this concept,
and use it as a lens to look at HCI,
and that's what we've been doing in this paper.
We tried to capture the bread and butter of HCI,
the kind of work that you see here at the conference,
and to offer something more than just a lens.
We're describing our identity as a field.
We're offering a tool
for generating and refining research ideas,
and we also tried to be inclusive,
so tried to be inclusive
in the sense that we have fantastic work,
looking at fabrication techniques,
tinkering with different kinds of engineering methods.
We have fantastic works on design,
and we have worked on looking at sub-Saharan and ICT use,
so how can you be inclusive, and give value to that work,
without be naive in the sense that assuming
that any perspective is valuable
just because it's different.
Okay, a lot has been said about HCI in the past.
Here's a small gallery of some of these accounts.
Some of them come from real papers,
and some of them are in the PCS in the review forums.
From the problem-solving perspective,
they are basically fielders.
They are scoping our work.
They are saying you should not study this kind of problem,
or you sound not use this kind of approach,
or you should be pursuing this,
and this kind of outcome.
It's easy to come up with pretty powerful counter examples
to all of these claims,
and what happens with the problem-solving capacity?
Here's the key idea, is that we take the problem,
and the solution, and and we form a new unit of analysis,
called problem-solving capacity,
so we ignore that approach entirely,
and just look at the problem and the solution together.
Now we didn't invent this concept.
It comes from Larry Laudan.
He's a philosopher of science,
and he was debating for Expo Kuhn,
and he made this following point.
In appraising the merits of theories,
it it more important to ask whether they constitute
adequate solutions to significant problems,
than it is to ask whether that are true,
or otherwise justifiable within the framework
of contemporary epistemology.
Now we've been applying this and extended this for HCI,
and the rest of the talk is about what did we learn.
First, one important point.
What Larry Laudan means by research problem,
is not the familiar problem
as we use in colloquial language,
rather it's defined by inabilities
and absences occurring in our descriptions
and knowledge,
and in our constructive solutions as a field.
That's not the same as a design problem,
or a problem that a user might incur.
The first observation we did with Kasper,
is that you can collapse this crazy,
hyper disciplinary field into just three problem types.
It's real valuable, because you can start comparing
the contributions of work
coming from very different traditions,
and not being bound by these traditions.
What are are these three problem types?
Larry Laudan introduced two of them,
empirical problem type, and conceptual problem type.
Empirical problem type, we all know.
Conceptual problem type is about inter-relationships
within concepts, and just second order problem type.
Constructive problems is something that we added
with Kasper, and it is basically about the construction
of artifacts using, for example,
means from design and engineering.
What Larry Laudan also offers us
is that the quality of sub-types of problems,
going from an increasing level of fidelity,
from complete absence of description to refining,
and retuning these descriptions.
No, we can finally try to define
what problem-solving capacity is,
that is show that it's something more
than what you typically mean by problem-solving.
There are five elements of this.
First of all, significance,
that we're solving significant problems
related to human use of computers.
The significance aspect by the way,
is often grounded to outside of our field,
to industry, and society, and not necessarily to the field.
It's defined by things like how many people do we address
in our work, or how important matters
in those people's lives we address.
Then there's the solution capacity aspect that is defined
in terms of three criteria,
effectiveness, efficiency, and transfer.
That means that the solutions that we acquire,
basically the outcomes of our work,
are evaluated in terms of
whether they're solving essential aspects of these problems,
and whether they're solving efficiently,
meaning with as little resources as possible,
and whether the solutions transfer
from instances to another one.
With Kasper we added a fifth criteria called confidence,
which refers to these traditional concerns
that we have about the logic of argumentation,
or validity, or liability, and so on.
Now this is the definition of problem-solving capacity.
Now what you can do, is you can analyze your work.
You can analyze whole topics in HCI,
and that's what we do in the paper.
Just to give you an illustration on how this works,
and what the benefit would be,
I took two classic HCI contributions,
Tangible Bits, and Fitts' Law.
Normally you wouldn't even consider comparing them,
because they are so different.
The other one is constructive problem type,
and the other one is conceptual.
Now, what do we learn if we compare that
with the problem-solving capacity perspective.
First of all, we should look at the significance
of the problems that they're addressing,
and there Tangible Bits is better,
because if it would work,
it would revolutionize basically all of input
with the computer.
Fitts' Law, although it's also important,
it's constrained to pointing tasks.
Then we also have to consider the four other criteria
that are relating to the solution capacity,
and for the illustrative purposes,
I collapsed them into just one dimension here,
and how it looks like there,
is the Fitts' Law is stronger in that respect,
because it has more efficiency effecting this in transfer.
It offers the model with which you can actually work,
and solve problems,
whereas Tangible Bits has a vision
and a set of demonstrators
that are harder to then apply to different problems.
Okay, so this was just an illustrative example.
Now what we wanted to learn with Kasper
is what could we say about HCI as a field, as a whole,
and for that end,
we first had to define what it means
to define the field as a problem-solving field,
and it boils down to three criteria.
What are the problems that we're addressing?
What are the desired problem-solving capacities
that we're after,
and what is the achieved progress in our work?
This is a generic description of our field.
Now we made some cursory first observations about HCI,
first of all that it's easy to find examples
of all of these problem types,
and all of the capacities in HCI,
so we are really, truly, diverse field.
In a single paper, your might easily find
even two problem types, such as if you have a study
and a problem type together.
We seem to have high tolerance for risk,
meaning that we are able to accept papers like this one,
or a paper on the design fiction,
or visionary paper.
We also seem to ground significance
not only to our own research body,
but to society and industry.
I guess that most of you would approve with this account.
Then we want to learn more, and dive deeper into this paper,
so we analyzed the best papers from last year's kind,
and first made a couple of observations
on how they are written,
basically writing culture that we have.
It seems that the problems perse,
they are typically well-described,
whereas the capacities are often ambiguously described,
so we can improve in that respect.
The second observation was that almost all the papers,
there was only one exception, are addressing empirical
and constructive problems,
so where are all the conceptual problems?
We argued that we're missing conceptual work,
that is tied together for it's empirical domain,
and in the constructive domain,
and this allowed us to explain why this big hole
is happening.
The big hole does not manifest, that we are not progressing,
which is progressing in a particular way,
that could be called leapfrogging,
so whenever there's a change in technology or context,
our solutions that we devote earlier
don't transfer into that new setting.
If we were a more mature field,
we would have progress like this,
which would be initially a bit slower,
but be more robust,
against external events later on.
Okay, what you can use this problem-solving view for,
is to expose flaws in our belief system,
to the extent that you can articulate them.
For example, the claims that we started with,
and the problem-solving perspective
gives you two kinds of counter arguments
to many of them.
First of all, that they tend to ignore
this diversity of problems and capacities
we truly need in HCI,
and then secondly,
many of them focus on either the solution or the problem,
ignoring the other one,
and this creates an imbalance,
but we are not limited to just describing how things are,
this can be used in a reverse manner, if you will,
to generate ideas, and refine ideas,
so you can take the five criteria,
and you can generate heuristics or ideas,
on how you can improve your own work,
and we offer a table in the paper
that you can use as a starting point.
To demonstrate that it's of some value,
we analyzed Fitts' law as a field,
so it's a field that has existed for many decades,
and we could show that there are places
where you can improve,
there are basically goals that we can set
for Fitts' law research that make some sense.
Of course problem-solving capacity perse,
doesn't say how those problems should be solved.
It's just saying that there's opportunity to improve
our problem-solving capacity in that domain.
Okay, to summarize,
we are offering a discipline-free view of HCI research.
That's a single concept,
so if you only have limited capacity,
like I have for attention,
then it's a nice candidate for describing HCI,
and I hope that I've shown than it's much richer
than solutionism, or your regular notion of problem-solving.
It also embraces variety without being naive,
without making the fallacy
that every perspective is valuable
just because they're different,
and it shows that we do have significant progress
in our field.
It's just limited into certain categories of progress,
and shows that we have a need to work
on conceptual problems,
that can tie together empirical and constructive efforts.
Finally, this is a full philosophy of science.
You don't need to buy into that.
You can use it also in a very pragmatic manner,
you can use these tables, and you can use this criteria,
to refine your research ideas.
Thanks.
(audience clapping)
- [Man] I'm very optimistic that there are some questions.
- Come on.
- [Woman] Hi, I know that you talked about this idea
about problem as being an expansive idea of a problem.
It seems like not all research is about solving problems,
so some research is about understanding themes or ideas,
and one example is a paper here,
at this conference by Steve Benford,
was thinking about an idea, accountable artifacts.
That's not something that's actually like
making a solution to anything,
that's understanding a space.
You're not going to get to a final point,
so how do things like that
fit into this idea or problem-solving?
- I didn't have time to go through all the sub-types
of each of the problems, but they always start
with the sub-type one,
which is basically complete absence
of a problem description,
so it could count as a contribution like that.
- [Woman] I'm not sure it's ever going to get
to a problem description.
- Sorry.
- [Woman] I'm not sure it's ever going to get
to a problem description.
That's not the point.
- What is the point?
(audience laughs)
- [Woman] Insight, revelation, illumination.
- That would count as sub-type one.
- [Man] Hi there.
Thanks for the talks.
Sabastian Beting from the University of Fjords.
I'm very much a pragmatist,
so this kind of philosophy of science there to me
very much smells of pragmatism, is right up my alley,
my question is then, as a pragmatist,
how does this change practice,
and not just the way we talk about things,
so how does this is yet another way in which we then
write our introductions,
and our papers slightly differently,
in order to justify whatever it is we write now,
or are interested in,
or whatever it is that we got money for,
but how is this actually changing research practice?
- I don't think there's any fast way to change the culture,
so it starts with individuals thinking differently,
and then that seeps into their practices,
and if they're bettering their own work,
then others will copy them,
and maybe it will distribute like that.
We are using this in our lab,
and it seems to work there.
- [Man] Okay, so the proof is in the pudding
that your lab will be so much humongously more successful
that everybody will follow on the bandwagon.
- Now we have 300 people in the room,
so let's see next year, right?
- [Man] Okay, thank you.
- [Man] Very nice work.
You've abstracted away the method,
and I understand why you abstracted that away,
but of course, in order to judge the solution,
we need to judge what the method was,
so can you talk a little bit more about that distinction,
like when does the method come in,
or is it the right suitable method for a particular problem?
- One of the criteria was confidence,
so you would evaluate the method
in terms of how probable you hold
that this problem-solving capacity that you claim holds,
so then you would evaluate basically
the methodology of it.
- [Man] So it would be part of the evaluation
of the solution then, right?
- Exactly.
- [Man] Yeah, okay, good, thanks.