(light electronic music)
- Hi, everyone.
My name is Mike Sinclair.
I am a Principal Researcher in the EPIC Lab
of Microsoft Research located in Redmond, Washington.
Our function is multi-fold.
A, is to connect with our product folks
and find out if they have any immediate technology needs
that maybe they can't address,
or they don't have the skillsets.
But the major function is to look ahead one year,
five years, 20 years,
where do we see Microsoft products going?
How do they serve the people?
How do they serve the public?
How do they increase quality of life,
the fun in life, their productivity?
Our lab specifically looks at HCI, Human Computer Interface.
How do we more naturally communicate with computers?
And that's both directions.
You may think that looking at a screen
is the computer's output and the keyboard
is the computer's input, but it goes way beyond that.
That's how we got interested in haptics.
We don't realize it, but we touch everything.
So we want to feel the texture of things, the compliance,
and all the physical properties of a device or an object,
comes into play when you're in a virtual world.
And as you know, in VR,
you're wearing a head-mounted display.
And you reach out and there's nothing out there,
other than maybe a re-embodiment of your hands,
but you get this virtual world.
Well, what's obviously missing is I can't reach out
and touch it with my haptic sense or my sense of touch.
And that's where we come into play.
We have some amazing scientists, engineers, programmers,
and designers that help us with augmenting
that sense of touch.
As part of a team, I have some amazing collaborators
in scientists, researchers, developers, and designers.
One of those is a fellow by the name of Dr. Eyal Ofek.
- Hello, everyone.
My name is Eyal Ofek.
I'm a Principal Researcher
at Microsoft Research Lab in Redmond,
in a group called EPIC,
which is Extended Perception, Interaction and Cognition.
And my interest in research is around virtual reality,
augmented reality, and as a result,
in rendering haptics, the touch sensing,
for those experiences.
And our vision looks at new areas where people
might be using virtual reality
and augmented reality in the future.
And one of those areas is the work desk.
We anticipate that people will use small
virtual reality glasses as a replacement
for monitors of today.
So, I could go anywhere I want,
whether it is working from home or working while traveling,
and instead of carrying a big laptop with a big screen,
I just put on something that looks like regular glasses.
And suddenly I see a large set of displays in front of me.
The fact that we are augmenting their visual display,
give us a new opportunity to also augment their touch sense,
so that they could actually work with virtual objects
on their work desks.
People can actually touch and feel models, objects,
terrains, remote patients,
and other objects that they need to manipulate.
And looking at this vision,
we came out with different solutions.
The first one is what looks like a small device
with nine actuators that can go up and down.
Now, this is actually representing what we see
as a haptic mouse.
What do I mean by that?
I could put my hand on the surface,
and the surface could be representing some virtual object,
maybe a terrain model of some area,
maybe a patient on the table,
maybe an animal that I pet, and so on.
And as I move my hands around the table,
the same way I do with the mouse,
this surface will change its geometry to represent
a larger object that I'm actually touching.
This surface can go up and down.
Each one off the actuator can change its geometry.
Now, one important issue is generating the material
for the top of the surface.
On one hand, it should feel very rigid,
like a solid object that I can touch.
I don't want it to be like a balloon
that if I touch, it's bending.
On the other hand, it need to be very flexible
so that if one side is going down and one side goes up,
it could stretch as a result.
And a major part of the development was
developing this complex material.
So this represent one way that we could feel haptics
by moving with our hands.
- It allows me to communicate at a distance
in natural user interface, NUI, that's what we call it.
And why should I have to conform to what only
the computer has to offer,
but why can't it be more natural like this?
One of my colleagues, Eyal, said,
"Why don't we put together kind of a game
where a child can put their hand on here,
scrub it around, and they are actually feeling a dinosaur?
And maybe in their virtual reality head-mounted display,
they actually see the dinosaur.
They reach forward, they touch it, they're moving around.
They can feel it breathe.
They can feel the skin ripple when you first touch them."
So, imagine how real that would be by adding haptics.
Another application in the realm of haptics,
are what we call discoverable haptics.
Notice that when I want to feel something,
what haptics is all about, I only need to have things
at my fingertips or in my grasp
or whatever I'm trying to touch.
I don't need to worry about anything around that,
'cause I'm not touching it.
I'm touching it with my hands.
So, with that in mind, why not have at your beck and call
little robots that run around on your desk
and the system through cameras, other things,
know where I'm about to reach.
And if the robots are fast enough,
the robot can run out there just in time for me
to actually touch whatever I think I'm about to touch.
And since the system knows what I should be able to touch,
the robot then conforms to that shape, size and position.
And in this case, let's say I'm investigating a cityscape.
And I come down on this roof of a house
and it's part of the gabled roof and it's at this angle.
So the robot would quickly go out there, orient itself,
raise or lower this little platform,
make sure it's at the right angle, at the right rotation,
just in time for me to touch it.
Wow, that's pretty cool.
Take my hand away.
And oh, I wanna touch this building,
and maybe it's at a different location.
And maybe you need more than one
or a swarm of discoverable robots.
And so, again, I don't need to cover the whole table
with all these pertinences, all these mechanics.
I just need to be there just in time to be able to touch
my hand or be there for my hand to touch it.
- A final prototype that we want to show you is the X-Rings.
This object is represent the geometry that our hands grasp.
Let's say that I'm in a virtual world
and I may pick up a bottle or I may pick up a cup to drink,
or I maybe pick up a handle of a tool to work.
Each one of them has different geometry,
which nowadays when people are using
one type of controller in their hand, all feels the same.
This device actually had a 3D shape display.
It has rings that can grow wider or narrower.
So, and do that extremely fast.
So, whenever I reach my hand to pick up an object,
this could generate the shape of the object
that I'm picking up.
And I can actually just by feeling it
can guess what was the object that I picked up,
whether it was a pint of beer or was that be the hammer?
And we see that as a completion to
touch object by encountering them
as we showed with the robots
or the moving shape display mouse,
this also enables us to pick object
and feel them as different geometries.
For more information about the future of haptics,
please visit these links.
Feel free to visit our Haptics Controllers and Rich Haptics
for Mixed Reality website,
or look at our latest announcement
in Microsoft Research blog.
You can also look at our latest publications,
and feel free to contact us.
We are happy to discuss this subject with you.
- We hope these brief presentations of our devices
and our view of haptics has inspired you.
Hope to see a lot of you in research
in the haptic topic in the future.
And thank you and goodbye.
- We have to actually rethink the entire AI stack.
- What is reinforcement learning?
Let's look at an interactive demo.
- [Man] All our code is now open source.
(light electronic music)