- But we also found that interface complexity
did not affect gesture recognition rates.
And so now Julia's going to talk
about the experimental design.
- Thank you.
Okay.
We first asked them to fill out a gesture sheet.
This is from the previous study.
Anthony et al ITS 2012.
This is an example of a seven year old participant
who filled out the gesture sheet.
This was especially important for children,
because sometimes they would forget
how to draw each gesture
while performing the task,
so they could look back at the gesture sheet,
and have an idea of how to draw each gesture.
So after completing the gesture sheet,
we then would ask them to do the tasks.
The first ones are the target tasks,
the one in the left is the abstract task,
and the one on the right is the complex task.
You can see the differences in the interfaces,
the one on the left is very simple
with the blue square,
while the one on the right has an animated penguin,
and it has an animated fish in a block of ice.
So when they would touch the fish,
the ice would melt.
The ones below are the gesture interfaces.
The one on the left is the abstract gesture interface,
which is again, very simple, with a white screen
and they would just draw with a black line.
And the one on the right has the complex interface
where each participant was allowed to choose
the line width and the line color before
performing each task.
After they completed the four tasks,
they then would achieve a prize.
And this is based on related work from
Brewer et al IDC 2013.
Where they looked to how to increase
the children's attention.
They added the score and also the prize at the end,
and completion rates went from 73% to 97%.
So we included this in the design of our study.
In our study, we had a total of sixty participants.
30 adults, 30 children.
We balanced gender,
and out of our participants 78% were right handed,
and 12% were left handed.
This shows the target applications with interfaces,
the one one the left being the abstract interface
where they would just touch the blue squares.
There's four different sizes
that would appear in different locations.
The one on the right shows the complex interface
with the animated penguin
where they would touch the fish,
the ice would melt,
and the score animation would appear.
This shows the gesture interfaces,
the one on the left being the abstract interface,
and the one on the right being the complex interface
with the animated bluebird.
And when they would finish,
the score animation would also appear.
I will now talk about the target interaction results
that we found in our study.
We first looked at holdovers,
where we found there was an effect
of interface complexity.
Holdover's defined as a touch
in the previous target location,
as the picture in the left you can see
there's still touches where
the previous target was located.
Out of our 16,000 touch events,
1.7% were considered holdovers,
and this was also similar to what was found in
Anthony et al ITS 2012.
Out of the holdovers,
the majority are performed by children.
92% of the holdovers in our study were
performed by children.
And what we found was there is a beneficial
effect of complexity.
The amount of holdovers went from a couple of hundred
in abstract interfaces,
to only a couple of dozen in the complex interfaces.
And we believe this happened
because due to the visual salience
of the ice melting.
It provided that visual feedback to the children.
The next thing we looked at was target misses,
where there was no effect of interface complexity.
We first looked at overall misses,
which we computed being the
percent of targets missed on their first attempts.
Previous work by Anthony et al ITS 2012
found that adults miss less than children,
and our results replicate that.
We then looked at misses by target size,
being as we had four different target sizes
for the targets,
and what we found was that participants
missed significantly more
on the smaller targets
compared to the larger targets.
The orange bars are the adults
and the blue bars are the children,
so you can also see that adults miss
frequently less than children.
We've also found a threeway marginal interaction
between target size, participant type,
and complexity.
What we found is that children
miss smaller targets more on the complex interfaces,
but missed the bigger targets less
on the complex interfaces.
So we found due to the presence of animation,
it was hard to locate the smaller targets,
but a lot easier for children to locate
the bigger targets due to the animation.
The next thing we looked at is edge padding,
where we defined edge padding as being
ten pixels for our study.
You can see as in the picture on the left
is the abstract interface,
and the one on the right is the complex interface
with our targets,
with and without edge padding.
What we found was that our participants
missed significantly more with targets
on edge padding.
The proportion of misses nearly doubled,
and this was also found by Anthony et al ITS 2012.
And we found that there was
no effect of interface complexity.
We also looked at touch pressure and size,
where there was an effect of interface complexity.
Participants had a lower touch pressure and size
for complex applications,
as in the interfaces.
And this might have occurred due to the
immersion of the game experience,
the participants were trying to capture the fish
and make the ice melt
for the penguin to catch the fish,
so they were more frequently to touch
the target's less with less touch pressure
and size than just touching a blue square.
The next thing we looked at is the location
where we found there was no effect of interface complexity.
We broke down the locations into 13 different locations,
the red squares illustrate where none of the targets
appeared on the interface,
this was based off of Anthony et all ITS 2012
where they did an informal app survey
locating where normal target locations appear.
So the red squares illustrate a location where
a lot of targets do not frequently appear.
We then looked up the left side of the interface
compared to the right side of the interface,
and we found there was a significantly higher
proportion of misses on the right side of the interface.
This may be due to the participants being
majority right handed,
and the way they held the phone
was mainly in their right hand
using their right fingers.
We also looked at the bottom of the interface
compared to the top of the interface.
We found that there was significantly more misses
at the top of the interface.
This may be due to the increased presence
of animation.
The penguin would blink,
and it was harder to locate targets
on the top of the screen
due to the presence of animation.
We then looked at response time
where there was an effect of interface complexity.
Response time being defined
as the amount of time it took the participant
to touch the target after it appeared.
And the important thing to notice,
and to mention,
is that one of the main factors
why the complex interfaces have a much higher response time
is due to the effect of animation.
But the important thing to notice in the graph
is that the younger children
had a significantly higher response time.
More response time than the adults.
So it took them much longer to locate the target
and touch the target than the adults.
And now Alex will talk about the
gesture interaction results.
- For the gesture set that we used
for our gesture experiments
was from Anthony et al ITS 2012.
It's some simple letters, numbers, shapes,
and symbols that children would have
likely been exposed to in school.
The gesture set contained 20 gestures.
We had them each complete 20 repetitions
of each gesture,
and for two apps, one complex and one abstract.
That gave us a total of 14,400 gestures.
So for our gesture recognition experiments,
we ran them in two different ways.
One was user dependent,
which is a within user approach.
And one was user independent,
that's between users,
and I'll talk about those more in just a moment.
We used the PR recognition algorithm
from Vatavu et al, ICMI 2012.
So here's an example of a user dependent experiment.
Say we have three users who each provided
an A Q heart
gesture,
and two examples of each one.
And we have a recognizer.
For user dependent we'll pick one user,
say user three.
We'll take one of the gestures as a test,
and then we'll take a set of trained gestures.
Given those inputs,
the recognizer will compute what it believes
the test gesture to be
and produce a result.
And if that result matches the test gesture,
that's a correct recognition,
and if it does not match, it's incorrect.
And then we repeat that for each gesture
for each user in our experiment.
For user independent,
it's a similar setup,
but instead we train on users who are not
in the test set.
So we take a test gesture from one user
and we take a train set from other users,
and we run that through the recognizer
just as before,
and we get a result.
The reason that we run these different types of experiments
is for user dependent,
we get an idea of how good we can expect
recognition to be,
given that it's trained on the same person.
Whereas user independent gives you an idea
of what kind of recognition rates you can expect
out of the box when a recognizer
has not necessarily been trained on the person
who is going to be using it.
So in our experiments,
we found that there was no significant effect
of complexity on gesture recognition rates.
However, we did see that there was
a significant effect of age on recognition rates.
The youngest children had the
lowest rates of recognition,
and the general trend was that
for older participants,
recognition rates were higher.
For user independent,
because these experiments require
larger sets of people,
we grouped together some age groups,
so we saw that there was
a significant effect of age group on recognition accuracy,
and we also found in this case,
that there was no effect of complexity
on the gesture recognition rates.
We also looked at the effect of the number
of training examples and age on accuracy,
and in this case we found that
for the younger children,
the increase in accuracy for each additional
training example was much higher
than for older children and adults.
So if you look at the bottom line,
the blue one, for five year olds.
The slope is much higher than it is for that of the adults
and that's because adding additional training
examples increases the accuracy much more.
So children benefit more from additional training examples.
And now Julia's going to talk about
the design implications of our study
and our conclusions.
Our first design implication is to provide
salient feedback of accepted input
to prevent holdovers.
As we found in the holdover results,
the complex interfaces had
a much beneficial effect of complexity.
There was significantly less holdovers
with the feedback of the fish being touched
and the ice melting.
Our next design implication is to avoid small targets
at the screen edge,
especially for the complex interfaces.
We found at the top edge and right edge of the screen
there was significantly more misses,
and with complex interfaces,
it was harder to locate the small targets
due to the presence of animation.
And especially with the small targets,
it was hard for both participants
to find the smaller targets
compared to the bigger targets.
Our next design implication is to consider
the trade off between visual saliency and response time.
As we found, there was a much higher response time
due to the effect of animation
for both participants.
So just consider for the applications
what they would rather have
when designing applications,
a higher response time
or to have a more complex interface.
Our next design implication is to train
gesture recognizers for younger children
with more examples.
As we found, especially with the five year olds,
the higher the training examples were,
the more their gestures were recognized.
And as our conclusion with interface complexity,
it affected some touch interactions,
primarily regarding visual saliency,
and it did not affect gesture recognition at all.
We also found a generalizable design recommendations.
When you combine this work with the previous
side of studies by Anthony et al,
we have shown a set of strong design recommendations
that can be applied to
children using touchscreens in any context.
For our future work,
we'd first like to look at larger touchscreen displays.
We found that there was results regarding
the location of the targets
and the size of the targets,
so we'd like to see if this pertains
to larger touchscreen displays as well.
We also would like to characterize children's gestures
to improve recognition.
We would like to do a more in-depth analysis
of children's gestures to support better recognition.
And we actually have a poster on this
tomorrow and Thursday.
And the location's on the slide.
And as we saw, there was a much lower
gesture recognition for children.
So we would like to get children's
direct feedback and input
in designing interfaces for themselves,
focusing on error recovery and correction.
And that is our talk,
thank you very much.
We would like to thank the people
who contributed on the paper,
and our advisor, Lisa Anthony.
(applause)
Questions.
- [Leader] We have time for two or three questions.
Please line up orderly.
- There's, yeah, the mic.
- [Leader] Tori's in one of the shots.
- [Audience Woman] Okay.
Very good job on the paper and the presentation,
both of you.
The question is for you.
You explained that on the slides
that the more number of samples for the children
showed that they picked up the trend more,
and they had better accuracy,
is that right?
- Right.
- [Audience Woman] How do you explain that?
Why do you think you see these differences
between the adults and the children?
- I think this is due to the higher level
of inconsistency in children's gestures.
They tend to be more inconsistent
within themselves,
as well as between themselves.
So when you get a higher number of training examples,
there's more likely to--
It's more likely to have
more gestures that are more consistent
with each other.
So that helps the recognizer to perform better
when there's more examples.
Whereas adults are very consistent in what they do,
so adding each additional sample does not
necessarily help the recognizer much,
because it's very similar to the previous gestures.
- [Audience Woman] But the number
and the accuracy measures were
normalized for each group independently, right?
So the number of errors for the children,
even though it was higher,
but the accuracy that we got was normalized
based on their own category.
- So the recognizer was trained only on gestures
from that age group.
Is that what you mean?
- [Audience Woman] Yes.
- Yes, that's correct.
- [Audience Woman] So could we,
could we conclude that the learning curve
for the childrens is
less steep?
Sorry.
They learn faster than adults do?
From the data that you presented can be
induce such a statement?
- I'm not sure that we could make a statement
about the way children learn
based on this data.
I think it really just shows that recognizers
improve faster given more data from children.
- [Audience Woman] Okay. Okay. Thank you.
- [Leader] We thank the speakers again.
If you want to--