Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
The promise of virtual reality, VR is indeed truly incredible. If one day it comes to fruition,
doctors could be trained to perform surgery in a virtual environment, we could train better pilots
with better flight simulators, expose astronauts to virtual zero-gravity simulations, you name it.
An important part of doing many of these is simulating walking in a virtual environment.
You see, we can be located in a small room, put on a VR headset and enter a wonderful,
expansive virtual world. However, as we start walking, we immediately experience
a big problem. What is that problem? Well, we bump into things. As a remedy,
we could make our virtual world smaller, but that would defeat the purpose.
This earlier technique addresses this walking problem spectacularly by redirection.
So, what is this redirection thing exactly? Redirection is a simple concept that changes
our movement in the virtual world so it deviates from our real path in the room in a way that both
lets us explore the virtual world, and not bump into walls and objects in reality in the meantime.
Here you can see how the blue and orange lines deviate, which means that the algorithm is at
work. With this, we can wander about in a huge and majestic virtual landscape or
a cramped bar, even when being confined to a small physical room. Loving the idea.
But there is more to interacting with virtual worlds than walking, for instance,
look at this tech demo that requires more precise hand movements. How do we perform these?
Well, the key is here. Controllers! Clearly, they work, but can we get
rid of them? Can we just opt for a more natural solution and use our hands instead?
Well, hold on to your papers, because this new work uses a learning-based algorithm
to teach a head-mounted camera to tell the orientation of our hands at all times. Of course,
the quality of the execution matters a great deal, so we have to ensure at least three things.
One is that the hand tracking happens with minimal latency,
which means that we see our actions immediately, with minimal delay.
Two, we need low jitter, which means that the keypoints of the reconstructed hand
should not change too much from frame to frame. This happens a great deal with previous methods,
and what about the new one? Oh yes, much smoother. Checkmark!
Note that the new method also remembers the history of the hand movement,
and therefore can deal with difficult occlusion situations. For instance, look at the pinky here!
A previous technique would not know what’s going on with it, but, this new one knows exactly what
is going on because it has information on what the hand was doing a moment ago.
And three, this needs to work in all kinds of lighting conditions. Let’s see if it can
reconstruct a range of mythical creatures in poor lighting conditions. Yes, these ducks
are reconstructed just as well as the mighty pokemonster, and, these scissors too. Bravo!
So, what can we do with this? A great deal. For instance, we can type on a virtual keyboard,
or implement all kinds of virtual user interfaces that we can interact with.
We can also organize imaginary boxes, and of course, we can’t leave out the the Two
Minute Papers favorite, going into a physics simulation and playing with it.
But of course, not everything is perfect here, however. Look.
Hand-hand interactions don’t work so well, so folks who prefer virtual reality applications
that include washing our hands should look elsewhere. But of course, one step at a time.
Thanks for watching and for your generous support, and I'll see you next time!