- (mumbles) work presented with Kasper Hornabaek,
so Understanding Affordance, System State,
and Feedback in Shape-Changing Buttons.
- Thanks for the introduction and I'll be
presenting this work that's done with Kasper Hornbaek
from the University of Copenhagen
as part of the Ghost Project.
So, shape-changing interfaces that could
use physical shape change,
as means of input and output and this dynamic nature
gives them the possibility
to offer new interaction techniques
and most of recent work has been focusing
on developing and building those interfaces
in order to take advantage of this dynamic nature
and this have resulted in shape-changing mobile phones
pneumatically actuated interfaces and shape displays.
Although we're getting better and better
in developing and building those novel user interfaces,
still our knowledge of their user experience
is still limited.
In particular, we don't know well how users perceive
and react to shape change
and some work have been focusing on that.
For example in the paper, Is My Phone Alive,
the author's measured the user's reactions
of some videos of shape-changing handheld devices
where they changed systematically some
shape-changing parameters and measured the effect
on user's perceptions of urgency, (mumbles) and effect
and in another study that was a (mumbles) study,
the authors measured the participants' responses
to a set of shape-shifting artifacts, where those
artifacts manipulated volume, texture, and orientation.
And co-motion was a shape-shifting bench
where the authors used an in-the-wild study
in order to observe
the effects of using it in social situations and
contextual atmosphere.
However, the existence of these studies,
some of the core elements of user experiences are still
not well understood.
So, for example, dynamic affordances and feedback.
So, dynamic affordance is one of the key elements
of user experience and shape change
which was introduced (mumbles) using the term that
used dynamic physical features in order to guide the users
on how to use the system.
So, as an example that was mentioned at Inform,
a system could change from a button, to a slider,
to a 2D tracking surface
in order to guide the user
how to better use the system.
And that's actually what we focus on, on this work
so, we try to present an in-depth empirical user study
of affordances, system state, and feedback in shape-change.
And we do this through using shape-changing buttons.
So, basically we developed some shape-changing buttons
and we ran a study in order to understand
how users perceive and act upon those buttons.
But, you might be wondering, why buttons?
And, for this we have two reasons.
First, simplicity. So buttons is one of the simplest and
most ubiquitous UI elements,
and by exploring the user experience of shape-change
using buttons, we tried to reduce the complexity
and novelty effect that's usually accompanied with
shape-changing interfaces,
which would make us focus more
on the user experience of shape-change.
And the second thing that buttons encapsulate,
well, the three things that we want to study.
Affordances, system state, and state change feedback.
So, I got a good button design from a machine
to tell me how to use it, and
if the machine is currently on or off
and after I press it, what was the effect of my action?
So, hence, affordance, system state,
and state change feedback.
So, mentioning what we did and why we do it,
so now let me tell you
how we did that.
And we used in our study some shape-changing buttons
and we designed ten of those that mimic a horizontal button
and as you can see, the designs were simple.
We didn't have a radical shape change,
but we used some actuation in order to
have some shape-changing behavior that could
provide affordances and feedback,
and we implemented them using mechanical and pneumatic
actuations
which are the most commonly used in shape-change right now.
So in some of the implementation details,
so the mechanically actuated one,
we used a motorized fader that could be found
in some of the sound mixers,
and we added on top of it a 3D printed handle,
and we coated that with conductive ink,
so we can detect capacitieve sensing
and we use an Arduino board in order to control
the motion of the handle
and also for sensing the touch interactions.
And, for the air button, the pneumatic control system
consisted of some valves, where by opening and closing them
through five holes along the button
we controlled animation, or the flow of the air
throughout the button
and we can mimic some affordances and feedback using that
and for sensing touch, we used a piece of
zebra conductive fabric.
We attached some strips of conductive
and non-conductive pieces in order to achieve
separate areas of conductivity.
And also, we used an Arduino board for the sensing
and controlling the pneumatic control system.
And here, let me give you an overthrow of the ten buttons
that we used.
We used invite to touch buttons,
feed-forward buttons,
on approach buttons,
and animated feedback buttons.
And, we based our designs on previous work of shape-change.
So, for example the on approach interaction
that was mentioned in Animate Mobiles,
and also we got inspired from successful techniques
of affordances and feedback and other UI elements.
For example, vibration movements and feed-forward
that they are used in hand-held devices, the applications
and the UI of them.
And now, I'll walk you through each of those buttons.
So, the Invite to Touch buttons use vibration movements,
as you can see horizontal and vertical
in order to hand affordance of interaction to the user.
Feed-Forward uses small animation in order to
hint the direction of interaction for the user.
Where you can see on the left,
it kind of moves and gets back
and with the air one, there is a big bump and then
smaller ones in order to hint the direction
of interaction this way.
The On Approach buttons react to
the hand approach of a user
in order to hint and stimulate interaction.
And lastly, for the Animated Feedback, we had two styles.
Linear Animated Feedback and Decaying.
For the linear, we used a simple animation
where the button toggles from one side to the other
and this is in order to say that the button switched
from on or off, or vice versa.
And the more fancier one, the Decaying one,
we used it, for example, for the mechanically actuated one
that it shows the effect of a bouncing ball,
like the button is moving to the other side
in a bouncy way,
and the air one used fluctuating pulses in order to
go to the other side in the Decaying form.
And, here in the Animated Feedback, we used two styles
in order to see the effect of varying the
animation feedback on the perception of the
state-change feedback.
So, when the button changes from on or off
or vice versa, what would be the effect
of those animations as kind of feedback?
And, for the user study that we performed,
we ran the user study in order to obtain
the participants' perception of affordance system-state
and state-change feedback, as I mentioned before,
using the buttons that I mentioned.
And those perceptions were obtained via
Think Aloud protocol, where we focused on those specific
three things.
And, our assumption was that by doing this
we can learn more about the user experience of shape-change
using those shape-changing buttons.
So, the study design went something like this,
So we actually had 13 buttons,
so the ten buttons that I just mentioned,
and we had three baseline buttons where they were
non-interactive.
So, the first one was just a plain piece of fabric
with no handle at all,
and the next ones had non-interactive handles,
where they didn't respond to the interaction of the user.
And we used this as a baseline in order to see
what's the effect of the shape-change
on the users' perception.
And we had 15 participants, where six of them were females
with an age range from 23 to 34 years old.
And none of the participants had previous knowledge or
interaction with shape-changing interfaces before.
The study set-up was something like this,
where we had the two prototypes present,
but we had only one active at a time
where we allowed the users to perceive it
and interact with it.
And, the order that we chose was random,
and the study procedure went something like this.
So, first we made one of the buttons active,
and before the users could touch the button
or interact with it, we asked
is the button on of off?
In order to ask about the system state,
and then we asked,
how would you use the button?
In order to ask about affordance.
And then, we let them use the buttons freely,
while thinking aloud, in order to capture their commands
and thoughts about how they perceive and use the buttons.
So, hence, we used the three things that we want to see.
System state, affordance, and feedback.
And we used (mumbles) video recordings
to capture the participants' commands and interactions.
And, afterward we transcribed those videos, annotated them,
and analyzed those videos.
And with analysis, we'll be mentioning some of the most
prominent results that we got out of that.
And, the first thing that we noticed is that
affordance is hard.
For example, as mentioned in the previous talk.
So, vibration movements were good
at grasping users' attention,
especially that the buttons moved without
the users interacting with them.
But, it led to some misinterpretations.
So, not always it was the case that users thought
that those are invitation for movements.
However, sometimes they thought
that the button is kind of stuck, and doesn't want
to move to the other side,
or that the button is signaling an error,
that it is overheating or there is danger with the system.
The second thing we noticed is that Feed-forward failed
in hinting affordance.
So, most of the users didn't see the Feed-forward animation
as an affordance signifier.
Although, they saw it as a system state feedback.
So, they thought that this movement told them
that the system is always processing or handling data.
Also, when interacting with the Feed-forward motion,
so instead of following the motion as we anticipated,
actually we can see here in the video,
that the user is trying to restrict the motion,
or stop it to the other side,
instead of following it to the correct side.
So, they were actually telling
okay, now it's moving, but I want to stop the motion
instead of following it.
And, for the animated feedback, we saw that it enhanced
the look and the feel of the buttons,
especially the air buttons.
Users commented that it made it more tactile and better,
because it didn't just stop from one side to
the other
but it had some kind of a flow.
And we saw also that animated feedback conveyed information
about the process of state-change, not only that the system
changed from on or off, but the process
of the system going through this.
For example, if the animation was fast,
it means that the system didn't have any problem
going from one state to the other.
Or, if it was a bit slower, then the system had some delays.
It's some kind of analogy of a progress bar, for example.
And after mentioning those results, let's see
how those results could mean foreboder of perspective
of shape-change, or for future work.
We acknowledge that in our study we used
a pretty simple design,
we used toggle buttons.
However, it was so simple, and the interaction span was
so short, so it's just toggling between two states.
It seemed a challenging problem in order to design
good affordances and feedback for that.
So, you could imagine how would it be the case
for more complex interfaces.
And that raises the question that we need more
user experience studies, in order to better understand
how users perceive and react on shape-change.
And, hopefully this will lead to
some better design guidelines that would help us
design shape-changing interfaces.
The second thing that we noticed, is that
shape-change is not suitable for simple feedback.
The feedback that we wanted to convey using the buttons,
is on or off.
And sometimes we saw that shape-change overloaded
this feedback.
And, led to some interpretations and ambiguity.
And this raises the question of when to use
shape-changing feedback, and when to use a simple visual
like an LED or just a simple label.
And this is still not explored,
where and when to use what.
And, the third and last thing that we noticed
so, through these interactions, as I mentioned,
users wanted actually to stop the motion.
So, shape-changing interfaces could design some animations,
or motions that's based on stopping to interact.
So instead of following the motion, you interact
with the interface by stopping it.
It's like interacting with your alarm clock,
you interact with it through stopping it.
And, in conclusion, we presented an in-depth user study
of understanding affordances, system state,
and feedback of shape-change.
And, although we used a simple interface in our study,
the study revealed that the question on how to design
good affordances and feedback for shape-change
still remains a challenging question,
and needs further studies and a solution.
With that, I would like to thank you for attention,
and I'll be happy to answer your questions.
(Applause)
- [Audience Member] Hi, (mumbles) A very nice work.
Don't you think that there's kind of a mispremise here
that motion, which is the result of shape-change,
could be used as an affordance?
Because I think in static physical affordances,
it's almost always about shape.
And, don't you think that's why you got
the results that you got?
And also, you know, motion is very often associated with
anthropomorphic, like something is alive?
And I saw a lot of the comments were sort of about that.
So, if you had a traditional on-off toggle switch,
you know, you convey that affordance by sort of having
this see-saw effect. Right?
And how you can fit your hands to it.
And you're almost like counteracting that
by having this thing move, and that's why
people are trying to stop it.
So, to what extent, like I think the challenge is
in dynamic shape interfaces, you can't always suggest
a shape is an affordance,
because the shape may not exist yet, right?
So, what do you think in terms of future directions?
Like, in terms of examining more shape,
or showing different possibilities for shape somehow
so you can have many affordances, but somehow
the users would see all of them simultaneously.
- Thank you for this, and as you mentioned
we weren't concerned about how the affordance of the shape
- Could you speak into the microphone?
- We weren't concerned about the shape of it,
because it's pretty known, the shape.
But we will focus more on the animations that we can change
and this we use, because the shape-changing interfaces
afford this dynamic feature of having change.
So, we use these animations in order to see if they would
enhance the affordances and feedback or not.
So, we're mostly focused on the animations
and the changes, not on the (mumbles)
- Yeah, well my question is, isn't it a better direction
perhaps to go, for the shapes?
- [Second Audience Member] Or, maybe, could I follow up?
Kind of a related point,
just going off what (mumbles) was saying,
I wonder how much of the effects you're seeing are based on
the visuals of the motion alone,
and I think it would be nice to see a comparison
where you have just the same animations you have
with the same 2D shapes, or even looking at it,
yeah I guess the same 2D shapes on a display there,
and comparing that,
because it seems like a lot of the things,
it'd be interesting to know having it be physical
changes the motion, and how people perceive that.
So, maybe that gets?
- Yeah, I think lots of future work to be done. (laughter)
- Thanks for the questions
(applause)