- A talk, we have Per Ola Kristenson.
- Thank you, oh it works, amazing.
So, I'm giving the talk, but the true hero
is the first author here, Xiaochi Gu, a first year
undergraduate student at the University of Cambridge.
But he's in China figuring out how
to manufacture this thing, so unfortunately he didn't
have time to come and give a talk.
I'm going to present Dexmo and what we
are basically tackling here,
similar to what you've heard in previous talks.
It's basically how to provide a compelling
and sort of convincing force feedback to users
in virtual reality.
So we're asking a lot of previous approaches,
various items discussed in the notes and I'll refer
to the notes for the details.
But the gist of it is that they are not really
fit for purpose.
They're either bulky or they're expensive,
or they are not mobile, or ineffective.
And what we are trying to do here is to solve this problem
in a way that we basically we call it
active passive haptics.
So what you do is you basically have the user
wear a mechanical exoskeleton as a sort of a glove.
The gist of it is that once you're moving this glove
these gears are on ratchet wheels,
are going to be free rolling, okay?
So the exoskeleton just seamlessly follows you
and is very dexterous.
When there's a signal sent from the computer
that the user's finger in the real physical world
is actually holding on or touching an object
in the virtual world, then the exoskeleton
will lock the corresponding joints in place.
The way we do that is we have this little device
at the bottom here with two rotational sensors.
This is normally free rolling.
Once we want to lock it in place we have
a stopping slider here so it's a linear motion
but it's activated by the device.
The stopping slider will then block the ratchet wheel
and hence the user cannot physically
move the finger anymore.
So it's actually physically blocked by the exoskeleton.
Which actually, if you think about it,
makes a lot of sense because most objects you touch
are actually rigid in a physical world.
This provides a very, very rigid experience
because it actually blocks the joints.
Sort of a breaking approach, right?
We basically break the mechanical exoskeleton
at the appropriate point.
I'll show you a video in a second.
At the time of the paper we had a simplified tracking model.
Now it's a little bit more sophisticated,
but let me first talk about it.
We basically have rotational sensors attached
to each of the joints.
Then we get the sensing data and then we basically
compute the four kimematic exchanges
using the standard algorithms.
And then back calculate the position of his joint
relative to the base of the exoskeleton.
We track the MCP joints and then interpolate
the DIP and the PIP joints via linear regression.
So we basically collected a bunch of samples
from five representative users to track,
fit the linear regression model to the system.
Now why would this be a good idea?
I think there are four good solution principles
for this approach.
So one is it's effective actually.
Turns out we don't actually need very fine grained
continuous feedback in order to provide
a good enough force feedback experience
or immersive experience of VR.
That's our claim.
But second and very importantly this can actually
be mobile.
Because we are not using, imposing any active force
on the user we don't actually require much power.
So we actually have simple standard mobile phone battery
last for four hours using the device.
And the device can also have been relatively lightweight.
It weighs about 270 grams and that number can go down.
Because it's so lightweight it doesn't require much power.
It obviously doesn't require tethered connection
and it can actually have it used wirelessly
and carried with you.
Currently, the device can be use wireless already.
Another important principle is to keep things safe.
And this is a very safe device right?
Because all it's doing is breaking the joints.
If it, for some reason would fail,
nothing bad would actually happen.
You would just be free moving, right?
Other systems, mechanical systems
that use pulleys and strings might risk,
for instance, bending the joints of the user
and damaging the user, or injuring the user.
Also, very importantly, if you actually want to have
some real impact you actually have to be able
to build this thing.
And this can actually be made inexpensively.
It can be mass produced according
to manufacturing instructions because it only relies
on inexpensive rotational sensors
and the other parts can be done using injection molding.
This is an early demo of the device.
You can see the guy moving around.
Now there is a ball connected to the exoskeleton.
You can see there at the end his limb is locked,
his finger is locked in place by the exoskeleton.
That's essentially how it works.
It's very easy.
You put it on, move around your fingers
whenever your computer system or virtual wall system
is telling you to use his fingers,
should not be able to move through a virtual object,
the exoskeleton simply locks in place.
There's a case so it can be used
with Unity, for instance.
This is what we used for our little study,
which we shall talk about in a second.
Our claim is this is effective,
the accuracy is reasonably good,
the response time is reasonably good.
It's definitely mobile, it's definitely safe
and it can actually be mass-produced.
So this is just a note, so we didn't have
as much space or time to be frank,
to do a sophisticated study about exactly
how well the tracking, to give you very precise numbers
on tracking, et cetera.
We're doing that at the moment.
But we did do a small study to show you the power
of the STK and the system as it currently is.
So we created an archery game from scratch
using Unity, and recruited 20 participants
via convenience sampling.
And we have them basically doing a simply,
playful, immersive task.
They pick an arrow and they put it into the bow and left go
and basically try to hit targets.
We basically compared using Dexmo versus not
using Dexmo.
We didn't start with the science,
all of our participants were exposed to both conditions.
And what we noticed is that the error rate dropped
from 61 to 44 percent, statistically significant.
But what was more important is that people really liked it.
They really felt like it felt subjectively immersive.
And there were lots of enthusiastic comments
such as response.
This is basically Dexmo.
I want to show you a longer video here
while I close off.
So, it's a mechanical exoskeleton and it produces
what we call active passive feedback.
So rather than reappropriating physical devices
as we have heard before, we basically locked
the exoskeleton in place and therefore
provide passive haptics via an active mechanism,
a mechanical mechanism.
Our claim is that it is effective,
it's definitely mobile, it's safe
and it can be mass-produced according to
manufacturing instructions and be mass-produced
and sold at a sensible retail price.
If you want to know more about it,
how to buy it or talk to us, or you want to develop for it,
I would be very happy to talk to you after the talk.
Okay, thank you.
(applause)
- [Imiter] Hi, thank you for the talk,
that was very nice.
Imiter from Media Zurich, I was thinking back when I watched
your demonstration, what if you made the device
even simpler.
Let's say without any motors or anything,
just always limited to a very specific
sort of grabbing lens, like always block when users
about grabbing that much which to me seems applicable
in any situation.
For example, your bow and arrow application,
this hand would be about that much, and this that much.
Would that work?
Do you have any thoughts on that?
- Actually we have filed a bunch of patents
on basically every conceivable way to do this.
So I don't want to comment too much about it,
but there are definitely other ways to do it,
including to provide actually both discrete feedback
and actually continuous feedback for using
a little bit extra power.
So yeah, I can talk to you later about this.
- [Maddy] Maddy Osmentian from University
of Southern California.
Great talk by the way.
I just wanted to understand better how
the bow and arrow experience works.
I understand if you can like the grabbing notion,
how it can provide that feeling.
But what kind of sensations are exactly you feeling?
I presume you're feeling the bow in your hand.
What's the other hand exactly doing?
- Yeah, it's a very thick string basically.
But what you're feeling, let me be very clear,
it's rigid feedback you're feeling.
You're not feeling any continuous feedback.
It's either on or off.
- [Maddy] So with my right hand, for instance,
when I'm pulling back am I feeling anything
in my right hand?
- No, you're not feeling anything then right.
- [Maddy] Okay, it's the left hand.
- Because it's not a continuous.
- [Maddy] Right, the left hand is feeling the bow.
- The system we used in the experiment
was not a continuous one.
We have a continuous one, actually at the moment.
But that was not used in this study.
You wouldn't feel it.
You would only feel if you were actually touching something
which is supposed to be rigid.
- [Maddy] But you don't feel like a stretching motion
or anything like that?
- No, you wouldn't, that's of course,
for continuous feedback.
- [DJ] I'm DJ from the (unclear) Lab,
and I have a speech impediment, but I'm going to try my best
to ask my question, okay?
Forgive me for asking a very blunt question.
But, I've been following Dexmo for a very long time.
We've been trying to order that mobile,
it has been in the pre-order state
for a very long time.
Can you comment on is it going to become
a public product or something?
- So, we basically have, I think about six versions
at the moment.
Basically, we want to, it's actually available
for pre-release for selected partners.
But basically we wanted to enable to do
a wider release we wanted it to be just,
to get the experience just right.
But we're over at the moment, so please be patient.
I'm really happy to hear you're a fan.