Hi, I’m Yi-Chi, a Ph.D. student at Aalto University.
I will be presenting our paper, Rediscovering Affordance: A Reinforcement Learning Perspective.
Upon seeing this object, what action do you think it allows?
Yes, the answer is press.
But have you ever wondered, how do you know that?
Usually, we attribute this ability to affordance.
Affordances refer to the actions available to an organism in an environment.
It is also a fundamental concept in HCI
because it explains how users know the right actions for each object.
For instance, you can tell a button affords pressing,
a handle affords turning,
and a dial affords rotating.
But how do we learn affordance in the first place,
and how do we adapt it in daily interactions?
Let’s take a look at the previous views.
For ecological psychologists, affordance is built upon body-relative features.
For instance, this object is pressable to you because
the dimension of the keycap somehow matches the dimension of your fingertip.
Also, the depth between the keycap and the base
compare to you is sufficient to be pressed down.
So you recognize it can be pressed.
The other perspective is from machine learning.
To them, affordance is a classification problem.
That is, you know this object is pressable
because you are able to classify it based on the visual features and details.
So there are two perspectives so far.
Yet, none of them explains how affordances are learned.
Or explains how we adapt affordance from interactions.
The theory we’re proposing is from a reinforcement learning perspective.
In a nutshell, our theory believes affordance is not innate
but highly adaptive and learned via experience.
By trying out interactive things,
people become better at estimating
what they can do with their bodies in the environment.
Now, let’s dive into our theory.
In our view, the core of what we call "affordance"
is an estimate people generate for "values" of possible actions.
People simulate motions in their minds
when seeing different objects,
and further estimate the value of these simulated motions.
Take a door handle as an example,
what are the actions you can think of?
Well, you may think of pushing it or swiping it.
If you want, you can also punch it.
But perhaps the most sensible one is to turn it
We can simulate these actions in our minds,
and place a value on each of them.
For instance, pushing the doorhandle requires force,
so the value might be -2.
And swiping over it probably causes nothing,
so the value is 0.
Punching it will make your fist hurt, so it’s a -10.
Lastly, turning it will give you an opportunity of opening it.
so the value is 10.
Because you’re able to generate the value of each action,
you are able to recognize what are the correct ones on the object.
The second principle is that affordances are learned
when the outcome, also known as the reinforcement signals,
are provided in response to motions.
Remember this handle?
We previously mentioned that you can generate the value for each action.
And the reason is we have interacted
with handles a lot of times in our lives.
Sometimes we did the right action, and received positive outcomes,
and some other times we might did wrongly, and received the negative outcomes.
These experiences gradually shape our behavior,
and let us know the most valuable action
should be performed here is to turn it.
Thirdly, there are two strategies of learning motions and affordances.
If the users fail, they generate alternative motions.
If they succeed, they keep using the same motion.
Take my keyboard as an example.
I know this object affords pressing
because I have pressed this keyboard many times before.
I also know that I can get
a good outcome by taking the same action, which is press.
Such a strategy is called exploitation.
A user can exploit the known knowledge
to achieve the best outcome efficiently.
However, sometimes we encounter new objects.
For example, a traffic light button.
What action do you think it allows?
Well, my first attempt is to touch the point where a hand is drawn,
and this is the outcome. It is a negative signal.
My second attempt is to grasp it and squeeze it,
and it leads to the same outcome.
Eventually, I found out I need to press the pattern,
and I finally acquired the positive signal.
Such a strategy is called exploration.
When facing uncertainty, we will attempt various motions
in order to acquire the optimal outcome.
But once we’ve learned that,
we can exploit the learned motions to be more efficient.
Our fourth principle is that affordance can generalize to unseen objects.
Let’s pretend it is the first time to see this kind of keyboard.
It is fairly different from other keyboards because it is very shallow.
However, you can still tell it is pressable, right?
The reason behind this is that
you have encountered so many pressable objects in your life.
You’ve learned to generalize your knowledge and find out
the right mapping between certain visual features to certain motions.
Lastly, humans are able to associate linguistic labels with affordances.
For example, you know this object is pressable
not just because you know how to create the motion,
but also because you know this action is named press.
While the linguistic labels are not essential,
it provides benefits in reasoning and communication;
especially communication.
We all have seen objects that we don’t know use.
For example, this round-shape object.
You can simply ask others,
and someone who knows will just tell you:
"route it and then press it."
Without demonstration, you can already get the idea
and simulate the action in your mind,
and further actually implement it.
Such as rotating it,
and pressing it.
To summarize our theory,
we believe that affordance is highly adaptive and learned via experience.
The source of affordance is the estimated values
of each motion that we simulate in mind.
We learned how to simulate these motions
and estimate values
by exploration-and-exploitation and generalization
We conducted two studies and built a robotic model.
Please take a look at our paper for more details
about the user studies and their findings.
Similarly, please read the paper for more details about the robotic model.
Finally, we’ve come to the discussion.
The essence of our theory is that
affordance is adaptive and learned via experience.
Our theory aims to establish a new common ground for HCI.
For example, the term has spread meanings in the field.
Our paper provides a concrete understanding
about where affordances come from.
Based on which, we can build more actionable discussions.
Also, there are various types of affordances in HCI.
Based on our theory, all kinds of them
are learned via the same mechanism,
which is reinforcement learning.
Finally, our theory answers what is a natural interaction?
In our view, even the most intuitive action is once
discovered, learned, and constantly adapted.
Natural interfaces are simply
the ones that the assumed motions
lead to the predicted rewards.
That’s all my presentation.
Please check out our paper and project page for more details.
Thank you.