OK, so hi.
I'm Wen-Jie from National Chiao Tung University, Taiwan,
and I'm glad to be here to present "FaceWidgets--
Exploring Tangible Interaction on
Face with Head-Mounted Displays."
And this work was in collaboration with Li-Yang
and Liwei from NCTU.
OK.
So FaceWidgets is a control interface
integrated on the HMD backside, which
allows the user to manipulate the controls directly.
And let me introduce our motivation.
As we know, freehand input has been a popular technique
for the head-mounted displays.
However, when the interaction time increases,
users may feel fatigue.
And FaceTouch allows the user interface on the HMD backside.
And it provides touch interactions.
And we believe the touch interactions
can be combined with the mid-air finger input
in the future interactions.
However, besides touch interactions,
we believe there is still some--
the design space on the HMD backside has yet been explored.
What if we want to interact with the physical objects
at this interaction space?
So we installed physical controls
on the HMD backside, which allows the user
to manipulate these controls directly with his hand.
And we do this because we want to enhance the HMD
input without hand controllers, and assess
the physical controls when you interact with the virtual one.
So this work introduces how we learned
to enable direct manipulation with FaceWidgets.
So here is an application of FaceWidgets.
You can see the user can manipulate these controls
directly with his hand in VR.
Our main principle is about direct manipulation.
We co-locate the virtual control and the physical control
on the HMD backside.
However, if you put the controls too close to the backside,
it will make the user feel eye strain.
It's difficult for eye convergence.
And we extend the control to a distance from the HMD backside,
which resolves this eye convergence problem.
And we can maintain the co-location
for direct manipulation.
So it turns out to be a lift in our final prototype.
We can use a lift to extend control from the HMD backside
and complete this interaction.
And now let's look into an example of FaceWidgets.
For example, the user is going to turn on the light in the VR.
So first of all, he will fixate his viewpoint on the light.
However, the button over here is within the disparity
of the light and the user's eye, so it will cause double vision.
And when double vision occurs, the user
might be confused about which control should he use.
So to address this problem, we use a rich canvas
with our interaction design to reduce
this disparity at the same depth to the controls.
So when the user fixates at the widget canvas,
double vision will be eliminated and the user can successfully
complete the task.
And after interaction ends, the widget canvas will fade out.
So let me see another video about our interactions.
The user uses the widget canvas reduce the double vision.
And here you can see the widget canvas,
and the user can use direct manipulation
on these physical buttons.
So let me show you about our implementation.
So our main purpose is direct manipulation.
And we have to extend the virtual buttons to a distance
from the HMD backside.
So this is our prototype FaceWidgets,
and you can see the physical controls are
mounted on the control panel.
And we have a lift driven by stepper motors, which
can extend the control, and the reflective markers
are for external tracking.
So here, the lift can extend these controls to align
with the virtual counterparts.
And also, we used the lift because it is foldable
and can reduce the obstruction and be more portable
when not in use.
So here is about our design, about control panel.
And you can see we can plug in the control panel
to the headers on FaceWidgets.
And also, to identify which control panel it is,
we use magnets on the control panel backside
and use [INAUDIBLE] sensors to detect which one it is.
So this is how we can do it.
We can reconfigure the control panel
and change the applications.
And since the Oculus CV1 doesn't have the tracking ability
on hands, we use VR gloves to provide external tracking
on the user's hand.
And for the OptiTrack, because our prototype
blocks the tracking of the Oculus HMD,
so we use OptiTrack to restore the tracking.
And all the tracked objects are calibrated
by calculating their [INAUDIBLE] transformation so
that all the tracked objects are under the same coordination.
So now I will show our interaction design.
And just as I mentioned, because double vision will occur
in our interaction, we use a widget canvas
to reduce the disparity so that the user can manipulate
our physical controls.
So this video shows the VR scene of the widget canvas,
and we blur and black the peripheral region,
and the fade-in and fade-out is controlled by the user's palm.
So this is the second interaction technique,
called palm-facing gesture.
And the user can use the palm-facing gesture
to invoke the widget canvas.
And now I will show some example use cases about our prototype.
So we designed two layouts about the control panel.
The first one is the fixed day out,
and the second one is contextual input.
And each of them is related to one application.
So in the fixed layout, each control maps
to one single function.
So we refer this application to the [INAUDIBLE]
and create a 360 video player.
So at first place, the user turns on the [INAUDIBLE]
switch, and he can get an overview of the environment.
And he can label the content inside of VR.
And also we provide a viewport knob,
which allows the user to turn his viewport
to another direction.
And he can use the reorientation button
to redirect back to the label content.
And what if there are multiple things
to manage in one application?
So we create contextual input.
So it's a small set of physical controls on the backside,
and we shall ask you to control different objects.
So here, for example, the user turns
on the light at first place.
And then he uses the same button to start the music player.
[MUSIC PLAYING FROM VIDEO]
And he moves on to the next one.
So in this application, the single control
maps to different module functions
depending on which one the user is interacting with.
And now we conducted three user studies
to inform our design of the prototype.
And the first one is the operation distance.
So here, we have to extend the controls
at a distance from the HMD backside, but at which
distance?
So here we study the minimum depth
that the user can still perceive the virtual controls
comfortably.
And the user has to adjust the depth of the cylinder
from 33 locations, which are evenly distributed on the HMD
backside.
So here is the test.
The user adjusts a cylinder at the HMD backside
to the distance that he feels comfortable and can
see the controls clearly.
And this is the other initial position in our test.
So let's see the results.
The overall average of these 33 positions
was about 12 centimeters, and the central region
is more flat than both sides.
So we decided to--
the blue surface shows the interpolation of our results.
And we added two standard deviations at the center
to 20 centimeters as our operation distance.
And we chose 12 positions evenly over this region
to place physical controls.
So in study, 1 we decided to extend the controls
to 20 centimeters from the user's eye position.
And now we're interested in whether showing hands can
facilitate our interactions.
So we conduct the second user study.
There is an independent variable, interface
with shown hand and no hand.
And the user has to rotate and press
a knob randomized from 12 positions in our test.
And we built a prototype which fits at 20 centimeters with 12
knobs.
And we use knobs to resemble the typical tangible interactions.
So our setup can reduce a part of the weight caused
by our prototype.
And we use OptiTrack to track the users'
fingertips and the headset.
So here's the test.
In the shown hand condition, the user
can manipulate these controls with his fingertips.
And in the no hand condition, the user
can only rely on his proprioception.
So in this study, we measured the completion time
and the error rate and offset distance.
And for each condition, we measured
the oculomotor or Simulated Sickness Questionnaire,
a subset of NASA-TLX.
And here are the results.
And we found out that displaying the hand
can significantly reduce the error rate and offset error.
And it can also reduce the mental and physical demand.
So [INAUDIBLE] using proprioception,
the user can operate face widgets,
but using hand reference can make the interaction more
precise and effortless.
So we decided to add the hand reference
in our final prototype.
But which type of hand model should we present?
So we conducted an exploratory study
to investigate the preference on hand models
and gather some subjective feedback.
OK, so we provided four hand models,
and the user had to experience four hand models
among two applications, which are fingertips and transparent,
silhouette, and realistic hand.
And we also asked the participants to rate
on these four usability questions.
And for more details, please refer to our paper.
And so, on these four dimensions,
we received some positive feedback,
and the user can operate the FaceWidgets, as well.
And on the preference, the users are most
like the transparent one among the first and the second place.
So the fingertips can provide precise control
but lack the sense of presence.
The realistic hand has a high sense of presence,
but some occlusion.
And the silhouette hand and transparent
can allow the presence of hand without occlusion.
So we chose the transparent hand in our final prototype.
And for the user feedback, they think our interaction designs
are intuitive and easy to use.
And one participant said that the rich canvas draws back
from the 3D environment.
So we think this can be studied in the future study
to improve the transition to usability.
So in the last study, we decided to display a transparent hand.
And now I will wrap it up.
So there are still some parts to improve.
And the total weight--
we want to replace the stepper motors with lighter actuators
to reduce the weight.
And also, we want to replace the parts of face widgets
into lightweight materials, like cardboard.
And we use external tracking of the hand,
and we hope in the future the inside out tracking
can provide hand tracking so that we
can enable our interactions.
Another improvement is on the number of the controls.
Because now the numbers of controls are limited.
And we hope to use haptic retargeting
to repurpose the virtual controls
into a small set of physical ones
so that we can increase the control.
So now I will present our contribution of this work
and conclude my presentation.
We have presented FaceWidgets, an HMD that
integrates physical control on the HMD backside, which allows
the direct manipulation in VR.
And we provided two interaction techniques, widget canvas
and palm-facing gesture, which can
enhance the direct manipulation on the HMD backside.
And finally, we conducted three user studies
to suggest the design space at this interaction space.
So thank you for your attention, and I'm
happy to answer your questions.
[APPLAUSE]
Thank you, Wen-Jie.
Any questions?
I have one to get started.
The examples that you showed were all
single-user applications.
Does it have any implications for multi-user interaction?
Yes, it is possible.
But we have to do is use some AR techniques to visualize what
exactly the function of the physical controls
and what the user is doing inside the interaction
so that the VR user can join the interaction of the VR user.
Any other questions in the audience?
We have time for a few.
I have another question I could ask.
Did you consider using, instead of
having the lifting mechanism, using like a redirection,
so a virtual hand that doesn't match the position
of the physical hand?
The virtual UI could be farther ahead while your physical hand
is actually here.
Yeah, but I think, in this context,
there would be some mismatch between the physical objects
and the virtual objects.
So we decided to enable direct manipulation so that what
you see is what you can touch.
That's our main purpose in this work.
OK.
Well, if there's no further questions,
let's thank the speaker again.
Thank you.
[APPLAUSE]