Generative theories of interaction
This article was published in the ACM Transactions
on Computer-Human Interaction in December 2021
by Michel Beaudouin Lafon, Susanne Bødker and me, Wendy Mackay.
First, what is generative theory, and how does it contribute to research in
Human-Computer Interaction ?
The natural and social sciences involve an on-going cycle between theory and
empirical observation.
Some sciences, such as experimental psychology and classical physics, begin with theory when
they present their research, which they then test empirically, often with controlled experiments,
the results of which allow them to revise their theories accordingly.
Other sciences, such as anthropology and astronomy, first descibe naturally occurring phenomena
that they observed, which they then explain with theory that is tested empirically with
other user populations or related phenomena.
HCI research clearly involves this back and forth cycle between theory and observation.
However, it also includes the design and study of novel artifacts, making it what Herb Simon
calls a “science of the artificial”.
We are not unbiased observers of independent natural phenomena, instead we theorize about
and observe what we design ourselves.
If we examine current human-centered design methods, we see that they focus on empirical
observations of users, either to inspire design ideas or to test them.
However, we also see an important opportunity for theory-driven design, which benefits from
a vast existing literature from other sciences --- about human beings, their interactions
with each other, and with artifacts in the physical world.
This paper introduces a methodology for creating Generative theories of interaction, which
we hope will not only inspire other researchers to generate new theories of interaction, but
ultimately, help practitioners generate novel interactive technology that is grounded in
and benefits from our collective understanding of human behavior.
So what is a generative theory of interaction?
We begin with existing theories from the natural or social sciences that are relevant to HCI.
These both inspire and ground the creation of HCI-specific concepts that are then operationalized
into actionable generative principles.
We apply three different lenses to these generative theories:
The analytical lens starts by observing how users interact with existing artifacts according
to a specfied set of generative principles, which allows us to understand and compare them.
The critical lens lets us identify identify problems or missed design opportunities
related to those principles.
Finally, the constructive lens helps us use these generative principles to come up with
new ideas and create new artifacts.
We illustrate how we have productively used a generative theory approach to develop novel
artifacts in our own research over the past decade.
The first example is instrumental interaction, which is based on two grounding theories:
human tool use and technical reasoning from paleontology and cognitive science.
The second example is human computer partnerships which is based on two grounding theories:
co-adaptation and the adjacent possible from evolutionary biology
The third example is community and common objects,
which is based on two grounding theories: activity theory and artifact ecologies
from psychology.
The generative theory of Instrumental interaction has, for example, the concept of an instrument,
which mediates a user’s interaction with an interactive system.
One of its key generative principles is ‘reification’, which transforms a command into an interactive
tool or instrument.
We can see how this works with StickyLines, which reifies the alignment command into a
magnetic guideline.
Here, the user creates a StickyLine that maintains the alignment of all objects
that are attached to it.
This alignment persists, even if one or more objects are moved.
Users can also offset objects from the line, for example when the geometric center does
not match the visual center.
They can then reify the offset into an instrument that can be copied and pasted and used to
offset other similar objects.
The generative theory of Human computer partnerships is built on the concept of co-adaptation,
where users must both adapt to an interactive system – they learn how it works – and
they adapt it to suit their own needs.
Two generative principles, discoverability and appropriability, can be combined, so that
users discover the capabilities of the system through progressive feedforward and feedback,
in order to redefine and personalize the system commands.
For example, we used these principles to design Fieldward, which lets users define gestures
that can be used as commands.
Users want commands that are easy for them to remember but must also ensure that those
gestures can be easily recognized by the system.
Fieldward provides a dynamic heat map.
As the user draws a gesture, the system changes color to indicate what it is recognizing:
if the gesture ends in a blue zone, the gesture is unique and can be adopted by the user.
If it ends in a red zone, it is not easy to distinguish it from another gesture, so the
user should continue or start over.
The third example of generative theory is Community and common objects.
A key generative concept is a common object, which acts as the center of activity for a
community of people.
And a key generative principle is : Malleability over time,
which supports the evolution of community practice and routines with respect to common
objects over time.
For example, the Caseline calendar is a shared timeline designed to help parents plan their
parental leave in Denmark.
The timeline acts to coordinate a set of activities and objects across a variety of people with
differing needs and interests that may change significantly over time.
Each of these generative theories of interaction occupies a different space within the scope
of possible HCI theories
Instrumental interaction is primarily focused on an individual user at a particular place
at a specific point in time.
Human-computer partnerships involve individuals or small goups of users, who may be co-located
or remote, and over longer periods of time.
Finally, Community and common objects involve large groups of users or whole communities,
located in many different places, over significantly longer periods of time.
But clearly, these three examples only touch on a small part of the large potential space
of theory for human-computer interaction.
We call upon our fellow HCI researchers to draw from existing theory in the natural and
social sciences, such as distributed cognition or boundary objects.
or to use theory generated from within the HCI research community
to create powerful new generative theories that inform interaction design.