- Hello, I am Da-Yuan Huang.
I'm glad to be here to present DigitSpace
Designing Thumb-to-Fingers Touch Interfaces
for One-Handed and Eyes-Free Interactions.
So authors of this project come from
National Taiwan University, Academia Sinica
and Keio University.
So, let's talk about on body interface first.
We know it transfers human skin into touch interfaces,
therefore, we could engage our fingers on the skin,
our ears and forearms or on our fingerpads,
or on other body locations.
So, it actually augments touch widgets on our fingers,
on our skin, and allows us to manipulate them by fingers
allowing users to perform always available
and eyes-free touch interactions.
So, the basic idea of this project is that
we found, we concluded a lot of advantages
of Thumb-to-Fingers Touch Interfaces.
So, Thumb-to-Fingers Touch Interfaces augment
the touch widgets on the fingers
and allow users to manipulate them by thumbs.
So, as you can see, the interaction process
requires only single hand.
And our human hands are dexterous to perform accurate
in a steady finger movement.
Finally, since the distance between our thumb
and the finger is short, the motion cost is also low.
So, previous works have demonstrated a lot of examples
about implementing Thumb-to-Fingers Touch Interfaces
by using a camera images, by using the changes
of the magnetic signals or by using
the changes of the muscle movements.
So, this already work are all amazing and great
and they mainly focus on the sensing techniques.
So, in comparison, our work aims to understand
how to design effective Thumb-to-Fingers Touch Interfaces
for one-handed and eyes-free interactions.
So, we therefore asked ourselves two questions.
The first is where are the comfort regions on fingers
to allow users to easily manipulate the touch widgets?
And the second question is that how the touch widgets
should be augmented?
What is the number?
What is the limitation and what is the proper layout
of this kind of touch widget?
So, the first question leads us
to our study one, comfort regions.
So, in this study, we will actually examine
the comfort regions for button and touchpad widgets
since the two kind of widgets represent
for discrete and continuous thumb-to-finger actions.
So, we firstly segment our fingers into 12 segments
and ask the participants to perform direct touch
and continuous actions on each of the finger segments.
So, the participants have to report their physical comfort
by following the five-point Likert scale.
And this is the result of our first study.
As you can see, these are the four distributions
of the physical comforts on fingers.
So, if you look at the distributions,
you will notice two main factors
that affect the comfort regions.
The first is the handedness.
So, according to our Wilcoxon test,
there exists a significant difference
between dominant and non-dominant hand.
And also, reported by the participants,
they think that their hand dexterity is very serious
in this interaction.
They might feel a little bit of uncomfortable
if they have to perform the actions
with their non-dominant hand.
The second factor that affects the comfort region
we think is the thumb anatomy.
So, as you can see, if you look at the distribution,
you will find that distribution's very similar
to previous works about using the smartphone singlehandedly.
And participants also report that they think
the muscle of the thumb and the joints of the thumb
are seriously constrain the possible thumb movements
during the interaction, therefore causing
different physical comforts of the finger segments.
So, in the first study, we ask ourselves
the first question.
Where are the comfort regions?
And we found that the handedness and the thumb anatomy
are the two main factors that might could affect
the comfort regions.
And therefore, based on the results in the following study,
we only asked the participants to perform the actions
with their dominant hands and we shrink
the interaction area in study two and three
by using the finger segments containing
the physical comforts higher than three.
So, now let's move to the second human factor,
the touch precision.
So, in the touch precision human factor
directly linked to the second question.
How touch widgets should be augmented?
For example, how the button widgets should be augmented
on the fingers.
So, we would like to know these questions.
So, we actually used a very brute-force way.
We proposed a four-button layout
with seven buttons in all on our fingers,
and assigned random targets on the user's finger,
and asked the participants to locate
the targets on the fingers in an eyes-free manner.
So, we utilized the air model for the position checking.
For more information of our position checking,
please refer to our paper.
So, I want to focus on the higher concept of our findings.
So, we have three main findings in this study.
After compute out of the accuracies of the button nails,
we have three main findings.
The first finding is that we found that
the accuracy decreased with the increasing number
of buttons, so this is totally understandable
since we believe that the tactile sensation,
the skin tactile feedback has its own limitation.
The participants tend to be more confused
if too many buttons were augmented on their fingers,
so this is understandable.
The second finding is that we found that
the accuracy significantly decay from the first
to the second finger segment which we believe
is also reasonable since in the study one,
the participants usually rate higher physical comforts
on the first finger segment than the second finger segments.
And it suggests that the inner thumb movement
might cause similar discomforts.
So, that's our second finding.
Our third finding is there's surprisingly
the diagonal thumb movement is actually does not affect
the accuracies.
We originally found that the motion efforts
of performing button-tapping on index finger
is more less on tapping on pinky finger.
However, the two way repeating their measure,
it is not the case.
So, now let's look at the last two findings.
So, we now notice that the inner thumb movements
might affect the accuracy.
However, the diagonal thumb movements may not.
So, we still believe that it is reasonable
because if you look at the thumb anatomy,
you will find the diagonal movements in this study
still locates in the comfort region
as we are using our smartphones.
So, we still would like to provide all of the participants
a proper layout of button widgets.
So, we would like to provide a general layout,
so we now want to examine each button layout,
and would like to understand how many buttons
could be augmented on each finger.
So, after our examination, we found that
for index and middle fingers,
you could augment five buttons on them.
And for ring fingers, you could augment four buttons on them
and pinky fingers, two buttons OK.
So, at least, we could provide each participant
16-button layout, which if you consider that
it would be really huge input power
and allow the participants to perform their interactions.
And if you would like to provide more input power
to the participants, we found another interesting phenomena.
So, this is the four-button layout of the index finger.
So, if you look at the distribution,
you will find that most of the participants
could perfectly devise the buttons
on the first finger segment.
However, the second finger segments,
the participant can not recognize all of them well.
So, this suggests that if you would like
to provide larger function sets,
a non-uniform button set might be more suitable.
So, now let's go to the study three, the final study.
So, in the final study, we select the stroke gestures
for the touchpad widgets since performing the gestures
in an eyes-free manner is very important research
in mobile scenarios.
So, we selected six graffiti letters.
These letters were considered hard to be recognized
and performed accurately in an eyes-free manner
as mentioned in previous works.
So, we used a high-speed camera in this study.
The participants have to perform the graffiti writers
in an eyes-free manner and the trajectory
will be calculated and computed by our system.
So, we designed three conditions for this study,
index condition, index plus middle condition
and cover condition.
So, the first two conditions, we would like to know
if the input area affects how accurate
can the participants perform the stroke gestures.
And for the last two conditions,
we would like to know if blocking the tactile cues
on fingers interfere the participants
performing the graffiti gestures.
So, the trajectory will be fed to the graffiti recognizer
and the whole system will compute out
all of the overall accurates or recognition rates.
However, the result is very disappointing.
If you look at over recognition rates,
it's only near the 60%.
And the One-Way Anova also suggests
that not only the input area, but also the tactile cues
are not used by the participants.
So, we began to wonder why the results are so bad.
So, we print out of the trajectory
performed by the participants and found that
the trajectory were usually rotated or distorted.
So, this, once again, indicates the importance
of the thumb anatomy issue.
So, not only the thumb anatomy affect
the possible movement, but also interfered
the possible moving directions of the thumb.
So, the participants are intentionally corrupted
or distorted or rotated their thumb trajectory.
Also, since the tactile cues were not used,
the misalignment problem is also very serious,
therefore, stand down these conclusions,
we will notice that rule-based recognizer
such as graffiti recognizer require random participants
to perform straight lines and wavy curves.
It's not suitable in the Thumb-to-Fingers Touch Interfaces.
So, maybe we should not to use rule-based recognizer,
but to train the model for the stroke path participants.
(coughs) Sorry.
But, if we print out of the trajectories,
you will find another interesting phenomena.
That is the writing behavior performed by the participants
are totally different, however the participants
seem to apply the same writing strategy
for the same graffiti writers.
The same phenomena also occurred
on graffiti writers D, O, R and P.
So, this once again suggests that instead of using
a general model to train by all the trajectory
performed by the participants, maybe a personalized model
might be better because the participants' writing behavior
differs one-by-one.
So, the results finally supports our assumption.
So, if we use the general model,
the participants, the recognition rate
can be boosted to 80%.
However, the personalized model could further
boost the recognition rate to 92%.
So, that's the conclusion of our final study.
If we would like to provide better recognition model
to the participants in Thumb-to-Fingers Touch Interfaces,
a personalized model might be better.
So, now I would like to conclude the three studies.
So, we would like to propose some design principles.
The first design principle is that we now know
that the hand anatomy is very important,
and not only affect the comfort region,
but also affect the layouts of different touch widgets.
Also, although the tactile feedback was not used
in the last study in the four-button tapping,
the participant still needs the landmarks
on fingers to recognize many buttons on the fingers.
Finally, the cross-finger touchpad is very important.
Although in our One-Way Anova, it indicates
that the input area is not related to the accuracy,
but participants still report that larger input area
provides more physical comforts.
Therefore, the cross-finger touchpad might be better.
Finally, since the button-tapping behavior
and the writing behavior between participants
are very diverse, maybe customize the button layout
or gesture recognizer will be better.
So, we finally would like to build a prototype
that includes all the design principles we just proposed.
So, we are inspired by one of our previous work.
The previous work is about using
the Hall-effect sensor grids augmenting the index finger
and try to compute the trajectory of the thumb.
So, we extended this design to our nail-ring chain model.
So, this is the final prototype we present in our paper.
It is a nail-ring chain.
So, the nail-ring chain composed by the interaction area
of this nail-ring chain consider the hand anatomy
and the comfort regions.
And we also preserved the tactile feedback.
The nail pads are composed
by a three-by-three Hall effect sensor grid
for cross-finger touchpads.
And the chain pad is composed
by a one-by-eight Hall effect sensor strip,
and for the customized button layouts.
So, once the user wear on the nail-ring chain,
they could perform button-tapping, finger-sliding
or cross-finger gestures.
OK, so we further built two prototype.
The first of two applications.
The first application is a lamp controller.
So, as you can see, the user could perform
this lighting finger-sliding to adjust
the amplitude or the color of the lights.
And the second application is a music player controller.
The user can write the song title
and play the song accordingly.
So, this is our conclusion.
In this paper, we conduct three studies
and we proposed the design principles
according to the findings of the studies.
And we would like to contribute this design principle
to the future designer of Thumb-to-Fingers Touch Interfaces.
So, this pretty much.
I would like to take this chance to thank
Dr. Yi-Ling Chen and Chin-Yu Chien
and also the Ministry of Science and Technology
and the Mediatek.
Thank you very much.
(applause)
And I'd like to receive any questions.
- [Daniel] Hi, Daniel Ashbrook,
Rochester Institute of Technology.
It occurs to me that the most comfortable thing
is maybe not always the best thing.
If you imagine that you might have a problem
with false positives, it's much easier for us
to touch the ends of our fingers accidentally
and so perhaps a further design principle
might be for uncommon operations or for seldom operations.
You might touch the inner parts of your fingers.
- Yeah, we thought about this question,
and finally we decide to use the double-tapping
as an event to trigger the thumb-to-fingers into actions.
So, maybe in the future, we could perform
like rotating our wrist a little bit
or double-tapping your other mechanisms
to allow users to trigger this event
without false positives.
- [Daniel] Nice work.
- Thank you.
- OK, so if there's no more questions,
that concludes the session.
Could we get a wonderful applause for all.