- Thank you for the introduction.
That was a very nice introduction for our work.
So, my name is DoYoung Lee from the UNIST at South Korea
and I'm happy to introduce our work about
designing socially acceptable hand-to-face input.
So, current mobile head mounting displays are quite amazing
with high quality screens and high performances.
Also because they are getting lighter and smaller,
we can easily imagine that these will be applied
to our daily life in the near future.
Then, how can you control these devices?
So, we can find various ways of using it.
For example, you can use the additional touch pad devices
or touch pad on the temple or simply you can put your arm
in the midair to make a comment.
So, although the AR works reasonably well,
they have some problems, right?
It can be cumbersome or it can have a limited input area
or it can cause some fatigue.
So many researchers are working on the
other types of the interaction surface, like On-Body.
So, in the SenSkin they average the deformation of the skin
for the interaction, and in the More Than Touch
they try to understand how the user uses
the skin as an input surface.
Although these are about the smart watches,
we can get some ideas right?
Touches to the body are expressive and easily accessible.
And the benefits can be extended to the hand-to-face input.
So, it's easy to access and usually there are no garments,
unlike the other body parts, and its proximate to HMD.
So, mapping between the control and the content
on the screen are more understandable to the users.
For example, upwards on the cheek should be the
same with the upwards on screen, right?
Even the touches to the face is quite common
and an obtrusive behavior.
So, from the previous research we found that it happens
fifteen times per hour up to fifty times per hour.
So, my researchers are already working
on their own face input.
Like the touching on the ear, or information on the cheek,
or the touching on the hair.
Because their face are quite a resting face,
they are all concerned
about the social acceptability.
But figuring out which face side, or which hand action
is socially acceptable remains difficult or unclear.
So we need a more structured method to access these things.
To deal with those questions,
this early work is those kind of tests.
So they gather the users' idea on Hand-to-Face input
and based on this research,
they designed some input techniques,
and they evaluated how comfortable each gestures are.
So as they research, they gathered many knowledge
about the hand-to-face input.
However, because they checked social acceptance
at the last stage, it was difficult to get
the general ideas to design a socially acceptable
hand-to-face input.
So we tried to expand this work in perspective
of the socially acceptability.
So we had three steps and each were
the elicitation study and case studies and validation study.
So in the elicitation study, we tried to understand
the users' idea on hand-to-face input in a social setting.
And in case studies, we developed two hand-to-face input
prototypes based on the data and design strategies
from the elicitation study.
And we also did some basic performance studies.
And at the validation study, we assessed
those input techniques and the design guidance
from the perspective of acceptability.
So first, the elicitation study.
So in the elicitation study is a user driven
gesture designing method.
So in the typical elicitation study,
we asked users to make a gesture for a certain task
like taking a photo and then they made a gesture
and they do evaluation for each gestures.
And in here, we combined social acceptability measures.
So per instruction, we asked them to make
a socially acceptable and subtle gestures
and for the gesture making, it was done in the big cafe
and there was a partner.
So these pair of the subjects made the gesture
in front of each other and they evaluate
the other's gestures in terms of the social acceptability
and the subtlety.
By using this method, we could understand their strategies
to make acceptable gestures.
So we had ten pairs of the subjects and sixteen tasks
and as a result, we got 320 gestures.
So first analysis was done for the face sites.
The left shows our research of the previous research
and the right one is ours.
So in here, we saw more emphasis on the chin and the ear
and we saw the lower emphasis on the forehead.
So we think this dramatic decrease in the forehead
is widely due to our social setting.
So users might try to avoid the face site like the forehead.
And in terms of the acceptability, ear, neck, and temple
was the high and hair and nose was low.
So this might suggest that away from
the center of the face are appropriate
for the hand-to-face gestures.
So for the second analysis, we categorized 320 input actions
based on the approaches to achieve
social acceptability and subtlety.
So two experimenters did these based on
the think aloud notes and the summary statistics
and questionnaire data.
So as a result, we got five strategies
with 237 gestures
in it and others were remained as unclassified.
Also, we found some gestures contains multiple strategies
like 58 gestures fell into two strategies
and eleven into three and three into four strategies.
So these are the five design strategies we found.
So the first three are about hiding input actions.
So miniaturizing was about making small gestures
like really small taps.
And obfuscating was about using hidden face sites
like back of the ear or back of the neck.
And the screening was about hiding the gestures
using the inputting hand.
So doing like this.
Next two are about avoiding arousing attention
even if actions are observed.
Camouflaging was about pretending on concise hand actions
like swiping the sweat or scratching.
And the re-purposing was about using clear and explicit
intentional actions like massaging the neck
or adjusting the hair or making some pose like this one.
So based on the elicitation study research,
we made two prototypes which are the ear touch
and thumb-to-face touch.
So these were on the most common face sites
which are the cheek, chin, and the ear.
And we accentuated design strategies we found
and we checked basic usability of the face touch gestures.
For the ear touch, we applied these three strategies.
So for the obfuscating, the ear was already hidden site
as it is somewhat away from the center of the face
and for the miniaturizing, we tried to make small gestures.
And in case of the camouflaging,
ear touch looked like scratching.
Something like that.
So this is our prototype.
So we used a backward facing camera on the side of the HMD.
So from the camera input, we used skin color
to segment the ear and finger shape.
And we did some basic performance studies.
So in the tap task, we had three techniques
and two target sizes, and six positions.
And we saw best results from
the liftOff and the large condition.
So this is the example of the best condition.
And the liftOff is a technique that activated
when the fingers are detached.
So while touching, the users can see the contact position.
And the large target size was- a large target was
the size of 1/6 of the whole ear length.
So in the best condition, issues 8.9% of error rate.
It took 2.5 seconds.
So previous research shows that 42% of average
for the same touch size.
So we think this is kind of substantial info
and the contact region.
The optical tracking was an accurate way
to detect touches on the ear.
And as future work, we will try to get various gesture
recognition like grasping or mid-air gestures.
For the thumb-to-face touch, we also applied
these three strategies.
So firstly, the touching with the thumb,
a thumb's nail naturally,
sorry, the touching with the thumb's nail naturally
screened the gesture with this hand.
So also, because the nail is quite small,
the gestures become small too.
So this is a kind of miniaturizing.
And in case of the re-purposing, it looks like
a thinking pose as designers do.
So this is our prototype, so we use three by three
capacity touch sensor array on the flexibility
with 0.3 millimeters of thickness.
So it was fit to the thumb's curvature well.
So we connected this piece from
the wrist mounted device and it was
wirelessly connected to the computer.
So from the touch surface input,
we extract the image moment of the biggest touch point
to calculate the properties like the centroid.
So in the tap task, we had three techniques
and two face sites which are the chin and cheek
and nine positions.
We saw best results from the dwell at the chin condition.
So this is the example of the best condition
and the dwell is a technique that activated when
the contact stays at a certain
position for 400 milliseconds.
So, while touching, users could see the contact position.
So in the best condition, it shows 21.5% of error rate
and it took 2.4 seconds.
Although we had somewhat high error rates,
we found the strong performance on the top left
and top right tip locations.
So it shows over 96% of the success rate.
Also, when you think about only edges,
we found that they were good at selecting each edges
especially for the top and both sides.
But they failed to select specific position on that edge.
So this is according to the softness
of the cheek or the chin.
So, while tapping, people are the whole edge
might contact the face because the nails are too small.
So as official work we will explore
the edge detection design to increase the performance.
At the last step, we did the validation study.
In here, we try to validate social acceptability
and subtlety of the input techniques
and design guidance.
And if has a similar form with the elicitation study.
So it was done in a public cafe and there was
a pair of users which is a gesture doer and observer
and there was six pairs of the subjects
and three input techniques for both devices.
And we skipped the data in here because of time
and I just like to show you the similar structure
of the interview research.
So we found some of our strategies were worked well.
These are the examples.
We found that camouflaging is like,
"okay this is similar to touching an earring
and I do this sometimes."
And miniaturizing was like, "too small to notice."
And obfuscating like, "since the ear is on the side,
it didn't matter."
And the screening, "hiding my thumb let others
do not notice this one."
And the re-purposing strategy did not
emerge in the interview.
So this is probably due to it's relatively
weak embodiment in the two prototypes.
So this is a summary of our work
and so we showed the elicitation study with the face site
analysis and five strategy analysis.
And in case study, we chose the two prototypes
with performance data.
And in the validation study, we chose the interviews.
We skipped a lot of the data from the papers
so if you are interested in this work,
then please refer to our work paper.
Thank you.
(applause)
- (Man) Fantastic. Thank you very much.
So this is from the Ulsan National
Institute of Science and Technology.
I thought UNIST maybe needs a little bit of explaining.
In South Korea.
And I'd like you guys to come up the mic
and ask any questions that you might have.
Yeah, somebody's on the run.
You can still make it.
Yeah, it's yours.
- (Woman) Thank you, my name is Susanne Boll,
University of Oldenburg Germany.
Sort of looking at the title of your work,
you're thinking of social acceptability
and you, of course, asked the individual on
their impressions, how they feel, can other people see it.
Did you also look at actual experiences of bystanders?
Because social acceptabilities comes not only
what I think I how I look, but also what other people
really think you look when you have your thumb,
for example, at the chin idea.
Did you also look at bystanders acceptability?
- Actually, that was why we used a partner method.
So because their partner with the gesture doer.
And actually, those two people are unknown each other
before the study.
So we think those partners can be a kind of bystander.
They're all.
- (Woman) But in the interviews, like in the validation,
you were just asking the person who used the gesture.
- Oh no, we asked to the other.
So this interview was to both of the...
- (Woman) Both, okay.
- Yeah, people.
- (Woman) And did you look at any kind of metrics
for social acceptability so there is also
a human computer interaction,
there's models describing a degree or level of that?
Did you already go into so far
or you'd say this is just the beginning
of understanding what is acceptable or not?
- I actually can be kind of the beginning of
the thinking of this kind of work.
But actually, in the paper, we have some data for
the social acceptability as some score.
So yeah.
- (Woman) And how well were you doing in the score?
- Actually, I just skipped that one yesterday.
(both laughs)
Because of the time but I saw that I have many time in here.
But anyway, yeah.
- (Man) You got a minute, you can show it if you want.
- Actually, that way to measure that kind of
acceptability was very similar to the previous, other's work
like, we asked them about the individual location
or in front of whom are you comfortable
or do you feel this is obvious or not for the subtlety.
So we asked those kind of things.
- (Woman) Okay, because it might be also different
if I'm at home with friends or if I'm in my office space.
- Yeah, exactly. In the club or in the street,
yeah that kind of question.
- (Woman) Thank you.
- (Man) I was struck by, you said unintentionally
we touch our face, head up to basically every minute.
Like fifty times an hour.
So that brings up the question, if you find these
gestures that are socially acceptable,
they are probably also the ones that look very much
like the things we do anyway.
Like you were saying, scratching your ear
or touching your nose.
So what about unintentional triggering?
Is there anything that you could recommend?
If I have to design a system that uses face touch,
what's your recommendation, how do I avoid designing
a system that gets lots of false positives?
Because every time the user scratches their ear,
they're sending a message or something.
- Yeah. (laughs) Okay, that can be a problem of
this kind of face touching or
the on body touching interactions.
I think maybe we can give some cues in the software
like, this is time for the touching like that.
- (Man) So that it's only actually actively
looking for these gestures when it's time to interact.
So that sounds like you would use it rather,
not to initiate something.
Like not send a new message but rather,
now is the time where you need to make a decision,
take a call, or reject a call.
- Yeah, actually our study was on just only
the performance study for now.
So maybe in the official work,
maybe we can think about that kind of points.
How can we activate our real gesture and not real gesture.
- (Man) Okay, I got lots more questions
but we'll take those offline.
Can we ask the next presenter to get set up
and we can turn off the projection.
(applause)
- Alright, so this was exciting.
Now, for a little bit of a change in pace