(upbeat music)
- Hello, everyone.
My name is Eyal Ofek.
I'm a principal researcher
at Microsoft Research lab in Redmond,
in a group called EPIC, which is Extended Perception,
Interaction and Cognition.
And my interest in research is around virtual reality,
augmented reality, and as a result in rendering haptics,
the touch sensing for those experiences.
Researchers, engineers, scientists, and designers,
has many years interest in haptics.
Among them is my colleague, Mike Sinclair,
who has an immense amount of experience in the area.
- Hi, everyone.
My name is Mike Sinclair.
I am a principal researcher in the EPIC lab
of Microsoft Research located in Redmond, Washington.
Our lab specifically looks at HCI, Human Computer Interface;
how do we more naturally communicate with computers?
- When we came out to think about haptics,
our focus was on handheld devices,
which are, at the time were rather rare
in the haptics research, and there's a reason for that.
A handheld device is rather small.
It cannot be very heavy.
It doesn't have a lot of mechanics in it.
And one major difficulty is that it is handheld.
So whenever it, for example, hit a virtual wall,
there's nothing that will stop the hand
from going through that wall.
So how can we represent such objects for virtual reality?
So what we brought here are several of our prototypes
that shows the thought that is streaming
from one prototype to another.
The first one here is called Normal Touch.
I can put one finger on a moving platform,
and that represents one point of a virtual surface
that I should touch.
Now, suppose I touch an object
and I want to move my hands further into the object.
What the device does is that as I put my hand forward,
that disc is pushing my finger backwards,
so I feel as if that touch point is fixed in space.
Now this has a limited range.
So beyond that range,
what happens is that my hand continue moving
into the object, but visually in virtual reality,
I see my virtual hand stay on the object surface.
And at one point when it's too far,
obviously the illusion will break.
This showed quite nicely, how people could feel
and manipulate virtual objects.
However, it's very limited in its utility.
So our next thinking was that if there's gonna be
a haptic controller that consumers will buy,
it should be able, the user to do a lot of things.
And here came another prototype called The Claw.
- And it consists also of a handheld controller.
In this case, it's got a wire back to the computer.
Normally if it's ever reached product stage,
it would be all battery operated.
And you can probably see that I'm moving it.
And I dunno if you can hear,
but there's a little motor
that is opening and closing on my finger.
And on command, if it knows where my finger is,
or where my hand is,
with respect to the object I'm about to squeeze,
am I picking up a grape or a marble?
Then as I close my finger in on my thumb,
it knows when to stop that.
Let's see I'm picking up a marble and stop it abruptly.
Oh, it's a hard object.
Now if I'm picking a grape, it knows when to feedback
with a little bit of force
and a little bit of compliance or sponginess.
Oh, it's something out there that's small and round,
but it gives a little bit.
And if I squeeze hard enough,
the system again knows it's a grape and it says,
uhuh, he squeezed way too hard,
and now we will burst the grape.
And the appropriate visuals will show.
And all of a sudden my index finger will come into my thumb
indicating from a force point of view,
I have squished the grape.
- This device was very successful
and generate very nice illusion,
and also enables people to do different tasks
in virtual reality.
However, like many haptic prototypes,
this is also a lab prototype that has a problem
to find itself into a consumer product.
Two major problems, not all of them.
One is the price of putting a strong motor
that could be strong enough
to overcome the power of human scale forces.
And the second is reliability.
When you have an object where you can put on the floor
and just stand on the hand
and just burn this expensive motor,
it's not something that you can put into a product.
So thinking of this usability, but the problem that it shows
we went out to another prototype.
This one is called the CapstanCrunch.
Visually, it's very similar to The Claw.
I could move my index finger and my thumb to grab things.
I could feel resistance if I pick up a virtual apple,
not to close my fingers.
But the magic here is that we are not using a big motor.
Instead, we are using a break called a capstan here,
which is powered by a really, really small motor
that hides inside the device.
And this motor is pulling a string here.
And a very small power pulling the string
doesn't enable me to close my hands.
So this device is similar in functionality,
but will be cheaper, more robust and more energy efficient,
which is important for a handheld device,
much closer to the vision
of having that as a consumer product.
- And then next in the evolution of our haptic device
is what we call TORC.
And the strange thing about TORC
is you notice there are no apparently moving parts
that are brought to the outside that would move my finger.
And so, we play on what we can make
the user think they're feeling,
and what they see
in their head mounted display overrides that.
And as I squeeze my finger against this plate
and these plates, as I squeeze, nothing moves here,
other than I'm applying more force,
but in the head mounted display,
I see my actual fingers move.
And this is enough to fool most people
to think they're squeezing something.
And as an output,
there are these programmable wide band buzzers
that I described before that as I squeeze,
it's not a single frequency buzzing,
but kind of a noisy, granular buzzing.
It gives me the feeling
that I'm actually squeezing something.
And one other difference in this device
opposed to whatever else we,
the other parts that we showed you,
is that it has this pad on this side.
And this pad is really a track pad
like you use when your finger
scrubbing a little pad on either your phone
or your notebook computer.
Only this keeps track of where my thumb tip is, or my thumb.
And it knows that my fingers are in the little detins
on the other side.
And it knows now that I'm doing the following manipulation.
And what I'll feel is one thing
and what I see in my virtual space might be a pencil.
And you know how you can manipulate a pencil,
even with your eyes closed.
You know how to manipulate a pencil or a key,
or other object, and do some pretty fine manipulations
that you couldn't do with other devices
in the haptic domain or even in the glove domain.
- Those two are representing a thinking,
which is a combination of both research.
How do we generate more realistic expressions
to people with the thinking of how do we make that
feasible to be a product for consumers.
- Right now, we're just at the inception
of technology of haptics, learning how to appease
and make believe,
not necessarily beyond a shadow of a doubt,
but enough fidelity there that,
"Okay I believe
I'm whole holding somebody's liver in my hands."
I think it's gonna be a long time
till we see a one-size-fits-all, if there is.
For more information
about what we see the future of haptics,
please visit the following links.
Feel free to visit our haptic controllers
and rich haptics for mixed reality site,
or take a look at our latest announcements in our blog.
You can also view our latest publications and videos,
and feel free to contact us,
we're happy to discuss these subjects with you.
We hope these brief presentations of our devices
and our view of haptics has inspired you.
Hope to see a lot of you in research
in the haptic topic in the future.
And thank you and goodbye.
- We have to actually rethink the entire stack.
(upbeat music)
- What is Reinforcement learning?
Lets look at an interactive chemo.
(upbeat music)
- All our code is now open source.
(upbeat music)