(silence)
- Thank you very much for the introduction.
My name is Santawat Thanyadit
from Hong Kong University of Science and Technology.
Today I would like to present our work on
desktop virtual realities using
mirror metaphor for natural user interface.
So, I would like to start with
a little bit of background,
why I work on this type of virtual realities.
Our lab is interested in using virtual reality
to improve office and classroom environment.
However, the head-mount display you have
isn't used that well,
and it does not scale really well
to a larger crowd due to its high cost,
and it requires special equipment.
Another important issue in head-mount display type VR
is that it is inconvenient for the user
to switch between different tasks,
for example, VR tasks and desktop tasks,
which is very common in this type of environment.
So we look into another type of VR
which is desktop virtual realities
also known as Fish Tank Virtual Realities.
This type of hardware is proposed
by Colin Ware and colleagues in 1993.
It consists of two main components:
the 3D display and the head-tracking.
The head-tracking information is used to generate
a 3D screen according to the user perspective.
So the user could experience the motion parallax
and the 3D effects at the same time.
There has been recent work on Fish Tank Virtual Realities
but most of the work, more or less, the same concept.
Of course, desktop virtual realities have
more suitable property for
the classroom and office environment.
This has relatively lower cost, less setup time,
and it is actually integrated well
into the work station because it's already have
a desktop in front of you.
The user can switch between applications
without removing the 3D glasses.
But, its downside is that it has lower level of immersion
and smaller screen size and smaller interactive space.
So, we tried to--oh, sorry.
Since the user cannot reach through the screen,
the past interaction in the desktop VR is limited
to the virtual pointings or extended virtual hand,
such as Go-Go techniques by Poupyrev et al.,
sorry, and colleague.
But, in this case, the user feel like
they interact with the objects using
remote controller, which is not very natural.
So, in our system, we try to create a
natural user interface in front of the desktop VR.
We use Kinect sensor to track the user head position
and Leap Motion sensor to track the user hand.
A 3D display is used for the user
to be able to see in stereoscopic 3D.
So, in our system, the system generates the scene,
based on the user head position.
The x, y, and z coordinates
are combined into a single coordinate system,
so the user's physical hand and their virtual hand
is aligned correctly from the user's perspective.
However, in this configuration we have an issue.
Since the virtual hand is aligned
with the user's physical hand,
the user could not interact with the objects
behind the screen because the user could not
actually reach through the monitor.
So, to utilize this virtual space, we propose a mirror
as an interactive metaphor.
But, why did we choose the mirror?
The mirror is used in our daily life.
We interact with objects through the mirror,
from simple tasks such as combing our hair
to more complicated tasks such as
doing the makeup. (chuckles)
Visual mirror is used in common issue
and (mumbles) issues alike.
For instance, our show trial on application allowed
a user to preview the course on customer bodies
to a virtual mirror.
For example, in the research, Bichlmeier
and colleague used the virtual mirror
to provide extra information in the surgeon simulator.
But, most of the previous work on the mirror is used
the mirror to provide extra information for the user.
But, in our case, we tried to use the mirror
as a 3D interactive metaphor, instead.
So, let's look at the video on how the mirror metaphor work.
In the mirror metaphor, we create the 3D screen
as a barrier between the front world and the mirror world.
If the user put the hand close to the screen,
it will show hand is transfer into the mirror world.
It will show hand as this way, like the flexion
of your real hand.
And, this way you can interact with objects
behind the screen.
If the objects is grabbed when you pull your hand close
to the screen, the object is transferred with the hand.
Since the system track the hand separately,
the user can generally experience that one hand
can interact with the objects in front,
and one hand interact with the objects behind the screen.
So, we perform a witness used, subject user study
to see how the mirror compare with the classic
Go-Go techniques.
This is the technique among, in desktop VR.
The reason we choose Go-Go technique is that
because it's have many similarity with the mirror metaphor.
We record 18 participant from our colleague
in our--your study, three of which are female.
All of the participant is between 21 and 49 years old.
13 of them wear optical glass.
Nine of them have prior experience in the virtual realities.
So, the task is we ask the participant to align the cube
in their hand to the flat (mumbles) cube
as correct as possible.
The participant have to alternately align the cube
in the front world and the mirror world.
For each technique, the participant had to go
to the training section and recording section.
Each participant have to align five cube
per section, in total.
In total, so each participant have to align
total of 20 cube.
We record composition time, partition,
and rotation accuracy.
To save some time, we summarize the key finding.
So, for further detail, please refer to our papers.
From the qualitative result, the mirror metaphor (mumbles)--
Sorry, in the mirror metaphor, participant
could actually control the optics position better
than the Go-Go technique.
This is shown on the graph on the left.
But, it's a problem when viewing with the orientation
in the mirror metaphor.
This is shown on the right.
So, another observation we saw is that there are
less chance of occlusion in the mirror metaphor,
during the object interaction that is
far away from the user.
Because when you should try to interact with objects
that is far away, the hand is actually far away
from the screen.
Another evidence of the mirror metaphor is that it is--
is used the same rate of control
with the user physical hand, so it is more friendly
to the physical-based manipulation, so that's (mumbles).
From the observation, there are also some downside
with the mirror metaphor as well.
It require some user adjustment, from the user,
especially when the hand is actually just transfer.
So, the user require some time to adjust to it.
It's also add additional transfer time to the system.
To give you an idea of the use case of the mirror metaphor,
we present a possible application scenario.
In this scenario, the user is working on organizing file.
The user can accurately pick the file
from the file combination locate in the mirror world
and transfer into the location which specified
in the front world.
The user could use natural user interaction
to interact with the virtual objects in front of screen.
When the user finish, they can put the file back
in the mirror world for the storage space.
So, this a simple application of technique
of different space and position accuracy
in the mirror metaphor provider.
In the end, I want to discuss
some possibilities for research.
In our lab, we are working on collaborative learning task
between two user.
It is our vision that the instructor and the student
can have their own private space.
But when the student applies some guidance,
the instructor can transfer to him,
using the mirror metaphor, to help the student
in the student space.
Another possible area of research is using
the PlayClaw, so the user can feel like they look
at the mirror, itself--look at the real mirror.
From our observation, there, the user feel
more natural, interacting with the virtual objects
behind a screen, when the PlayClaw is (mumbles),
even though the PlayClaw can actually penetrate
the virtual objects, in this case.
Thank you very much for listening.
I love to answer your question, now.
(crowd applause)
- [Man] Okay, we'll take some time for your questions.
- [Man] I am (mumbles) from the University of Saint Andrews.
I think it's very eloquent work.
So, thank you very much.
I'm wondering a little bit about the distinction
between egocentric and exocentric perception of 3D
because we know from the psychologists
that these are different. - Uh hmm.
- [Man] And I wonder whether what you're doing,
actually, essentially, forces the user to switch
from one to the other.
And, I'm not expecting, really, a precise answer
about these, but have you thought
a little bit about this?
What is your personal experience, using it?
Do you think that the brain, now, has to switch
to, kinda like, a different mode every time
I cross the screen boundary?
- Uh hmm. - Yeah.
- Okay, actually, we thought about it a lot
because we want to try to, with this mirror metaphor,
to use in a qualitative (mumbles).
So, we perform some primary experiment on this,
and the teacher--we have like, a setup
of the teacher and student.
So, in that case, when we used the mirror metaphor,
the teacher, when the teacher transferred the hand in,
the student--the hand will look as if the teacher hand
is actually over the student's.
So, it's more egocentric view.
And, that's actually more, feel more natural for the user,
for the student, instead of hand protruding out
of the screen.
But, from the teacher we applied, it's--
They have to require some adjustment for that,
but it's more advantage for the student, in this case.
- [Man] I think it, just as a note, it would be also
interesting to see what happens with personal space
'cause the psychologists have studied how, like,
which other details actually change that would we consider
personal space or middle space, et cetera.
- Hm, we rely, based on the percent of the screen itself.
So, we haven't studied on that.
Thank you very much. - Thank you.
- [Man] Do we have other questions?
Okay, I have only a quick question.
What happens if the user want to place the object direct
in the baseline between zero parallax and--
- Uh, yes, we still have problem on that.
It's depend on our implementation.
So, as you can see, when the hand transfers, it's use.
We set some time frame that the use, the hand is transfer.
And, we access, can be sets the range.
So, if currently, we set the range, like quite far
from the screen, because--not as far,
but around, like five centimeter from the screen
so that they can actually track the hand correctly
from, in that position.
But, if we acquire more (mumbles), the screen person
can be, like, this area can be minimized.
- Okay. - Thank you very much.
- [Man] Let us thank our speaker, again.