- Hello, my name is Hemant Surale,
and I'll be presenting Experimental Analysis
of Barehand Mid-air Mode-Switching Techniques
in Virtual Reality.
So, what does this term mode switching actually mean, right?
Let's dissect that and focus on mode.
So in order to understand mode, let's play a game.
And in the next slide, what's gonna happen is you will see
a pinch hand posture moving along horizontal axis,
and you will need to guess what this posture would do
inside a virtual reality application.
So are you ready?
What do you think this action created in virtual reality?
Well, it drew a line.
I'll give you another chance.
This time with a cue.
Try to guess what this hand motion created
in virtual reality or what it did.
Well, it translated the cube.
Now the last option.
You won't get more chances beyond this.
It should be the easier one.
What do you think this hand motion did now?
It rotated the cube, okay?
So the interface mode determines what user input
is mapped to an application action.
And mode-switching is simply a transition
from one mode to the other.
And in our work, this state information
is not just with the application,
but hand posture itself will maintain the state.
So, for drawing you can use pinch hand posture.
For translation you can use fist hand posture.
And for rotation, some other hand posture.
In past, mode-switching investigations
have primarily focused on mouse, pen input, and touch.
But given wide range of input techniques for mid-air input,
where is the mode-switching investigation for that?
So this is exactly what we do through our work.
And as a first step, we start with literature survey.
So all the past work which talks
about bare hand mid-air input,
we sort that work and filter that
through these three filtering criteria.
The first one is independent.
So a hand posture has to be independent.
What does that mean is it should
be fast to recognize and independent.
There shouldn't be a temporal component associated with it.
For example, drawing a circle in mid-air wouldn't be
a good hand posture for mode-switching analysis that we do.
The second one is kinesthetic action.
So user should be able to maintain the hand posture
throughout completing the task.
And the last one is unconstrained.
While participant or the user is maintaining
this kinesthetic hand posture,
he should able to do that around his body while being in VR.
Applying these three filtering criteria on this past work,
what we come up with is these six hand postures.
So pinching fingers, extending fingers, closing hand,
open hand, raise hand, and touching the body.
And the experimental task is line drawing.
In the baseline posture or the baseline condition,
you put your hand inside one of the spheres,
starting drawing line, and then connect that
to the second sphere and get your hand out.
And this is how the baseline task looks like,
where you use pinch hand posture to draw line.
Let's take a look at compound task.
In compound task, only difference is you will be drawing
red lines by using one of the mode-switching technique.
Here, it is fist.
And in order to calculate the mode-switching time
taken for, for example, let's say fist,
what we use is subtraction method.
So in the baseline block, you'll be drawing
all lines using pinch hand posture,
and in compound tasks or compound block,
what you'll be doing is drawing red lines,
alternating red lines using one
of the mode-switching technique.
So the top one is baseline block
and the bottom one is compound block,
where you are switching modes using one
of the mode-switching technique.
Now, the mode-switching time taken for that compound task
or the mode-switching technique, say, fist,
is simply the difference between the time taken
for block B and the average amount of time
a participant took to complete the block A and C combined.
So we conducted the first study with eight such techniques.
This was within-group study and we measure
task completion time, error rate, and subjective ratings.
The first technique is non-dominant fist technique.
Here as you can see a non-dominant hand fist
will act as a mode-switch.
In dominant hand fist technique
a dominant hand will form a fist to switch the mode.
In non-dominant palm technique,
orientation of palm will switch the mode.
In dominant palm technique,
dominant palm posture will switch the mode.
In dominant point technique,
pointing posture will switch the mode.
In non-dominant device technique, carefully take a look
at how user is holding the controller
in his left hand to switch the mode.
In non-dominant head technique,
a force sensitive resistor was taped
onto the side of the head-mounted display,
and user has to press that in order to switch the mode.
And we use we use force sensitive
resistor for these two purposes.
Beyond non-dominant head technique we also use
force sensing resistor for precisely detecting
when the line drawing task started.
And the last technique, non-dominant field of view.
This technique, as you might have guessed already,
by just bringing the non-dominant hand
in field of view will switch the mode.
Now these are the techniques.
Let's take a look at their mode-switching time footprint.
As you can see here, dominant hand techniques
performed poorly compared to non-dominant hand technique.
Non-dominant hand techniques,
particularly non-dominant device technique, is the fastest.
Let's take a look at the error rates now.
In terms of overall error rate,
non-dominant techniques are pretty comparable
to baseline hand posture, which is pinch,
and dominant hand techniques perform poorly.
And one of the reasons for this is participants felt that,
and these techniques are point, fist, and palm,
they are significantly different from baseline posture,
which is pinch.
So we suspect that maybe adding subtle hand posture
would reveal some different results.
And to summarize the results of the study one,
non-dominant hand techniques
were less error-prone and faster.
Dominant hand techniques, fist, palm, and point,
were perceived to be confusing.
And this set us the pace or the base for the study two.
And through study two, we particularly want to answer
'Can dominant hand actions more similar
to pinch trigger would perform better?'
And, 'What is the effect of using
a device controller as a manipulation trigger?'
So, in baseline task, you saw that
pinch hand posture was used to draw the line.
Can we use a controller and see
what is the comparison between those two techniques?
So, we conducted a second study, with four techniques.
We'll get to those four techniques really soon.
This was also similar to the study one.
We measured task completion time,
error rate, and subjective ratings.
So, first subtle form of dominant hand
technique is dominant orientation.
So, by just changing the orientation
of dominant hand, mode is switched.
In the dominant hand middle technique, all you do
is just use your middle finger to just switch the mode.
So, in the baseline tasks you use index finger,
and to switch the mode you would use middle finger.
Non-dominant hand device technique is similar to
what we saw in experiment one,
except that in order to draw the lines,
now user is holding the controller in his dominant hand,
and the trigger is used to draw the line.
And the last last technique is non-dominant palm technique,
where non-dominant palm orientation is changed.
And another change to this technique is user
is holding the controller to draw the lines.
And these are the exciting results.
So, as we suspected that maybe a subtle form of dominant
hand techniques will result in better mode-switching times,
and that's what we see here now;
that dominant hand middle technique
is comparable to the fastest technique
non-dominant hand device that we saw earlier.
Let's take a look at the error rate.
So, one of the wins of this second study is that
controller baseline is comparable to the pinch baseline.
And that's one of the strongest highlights of this study is
a pinch is a practical alternative to a controller button.
The second one is that non-dominant actions
for accuracy-focused tasks are more suitable,
and dominant hand actions are suitable for longer tasks.
Like, imagine, if you free your left hand
from doing the mode switching,
it can be used for some other purposes,
and dominant hand can continue to switch the mode,
as we saw in the second study.
Next one is, 'Avoid the pointing
posture for frequent mode-switching'.
This is quite surprising, given that pointing-posture
is such a common posture a lot of applications use.
Subtle dominant hand techniques are a promising alternative
to non-dominant controller techniques,
and switching between very different hand posture,
specifically dominant hand posture, may be confusing.
To summarize, we provide empirically-driven
insights to use bare hand mid-air input
in virtual reality applications.
Thank you; I'm ready to take the questions.
(audience applauds)
- [Man] So again, if you stick a hand up
in a SV we'll come over with a microphone,
or if you want to write it down
I could read it out, alternatively.
Okay, so, Hemant, thank you for your work.
Actually, while people are doing that,
I was curious about the, uh, you had force sensitive trigger
and stuff, and we were talking before and I said,
even going back ten years or so, to the Nintendo Wii,
people have a tendency, in my own experience, anyway,
to sort of fake technology in a way that,
they simulate it, actually, pretending
to be doing the actual thing, but actually,
they're not actually doing the right posture.
Did you see that in your work, or how did that,
how would you detect that sort of behavior?
- Yeah, that's actually a pretty interesting question.
So, it is possible that you can
overlap trigger and forming a posture,
but specifically for our study,
in order to control the overall, you know,
some movement for every mode switching technique,
we follow the sequential model.
What does that mean is that you form a posture,
and then in order to start drawing the line,
you use force sensitive resistor to trigger it.
And this way it is kind of like maintained,
like the same sequence of action is maintained
throughout all the mode-switching techniques.
So, results are, you know, like,
there's no biased results then.
Yeah, thank you.
- [Paul] Hi, I'm Paul Green from the University of Michigan.
How well did your a prior predictions
agree with the experimental results?
- Yeah, can you repeat the question, please?
- [Paul] Yes; how well did your a prior predictions
agree with your experimental results?
(murmuring)
- [Man] Actually, is really hard unfortunately, so...
- I think it's echoing in here, so I couldn't really hear.
- [Man] If you could talk a bit louder.
- [Paul] Right; how well did your a prior predictions
agree with your experimental results?
- [Man] How did your predictions
compare with your actual results?
- Do you mean specifically for the second study?
- [Paul] For both.
- Yeah, I might need more clarification
on what is 'a priori assumption' means.
- [Man] Like predictions before
you collected data, for example.
- So, for study one and study two,
both were like the mode switching techniques were picked
based on the classification or those filtering criteria.
So there was no specific prediction attached
before we conducted the study, or even to see the results.
It was purely driven by curiosity,
especially for the second study.
- [Man] Okay, let's give him a round of applause.