My name is Shahab and I will be presenting the work
that we did in Arizona State University with my colleagues
Frank Liu, Alireza Bahremand, Assegid Kidane, and Robert
LiKamWa.
And our work is titled "SWISH, a Shifting Weight Interface
of Simulated Hydrodynamics for Haptic Perception of Virtual
Fluid Vessels."
Natural liquids exist all around us and on a daily basis,
we interact with various types of liquids.
The feedback that we receive from each and every interaction
helps us in handling them properly.
It stands to reason that bringing these sensations
into virtual environments will help us
in the future of workforce training and also education.
In the emerging era of online education,
students can go to virtual laboratories
to mix acids and bases and experiment
with chemical reactions.
Bringing tactile sensations into these laboratories
will transform the future of education.
There has been a lot of work in virtual environments
to render realistic fluids.
But even though these liquids look real,
they do not feel real.
The chief characteristic of the liquids
is that they take the shape of the vessel that they are in.
When we pour water into a glass, the water
takes the shape of the glass itself.
As we start moving the glass of water around,
fluid particles and water particles
start moving around, subject to forces, such as gravity, also
properties of the water itself, such as cohesion, adhesion,
and contact forces with the glass itself.
As we move the glass around, we can feel
the center of gravity shift.
That is obviously not the case with the current VR equipment
because they have a fixed center of gravity.
But what if we could make it so that we could design vessels
for VR and AR that could feel like they
have virtual fluid inside them?
And then we would make the center of gravity
to shift, to make it so that they have volume and also
viscosity.
Prior work in this area has largely
focused on rigid body objects.
There has been work in providing tactile sensations of fluids
as well.
But most of these works have focused
on using grounded devices, devices
that users would have to go to to interact with the fluids.
Our work extends these works and provides an ungrounded device.
The key idea behind SWISH is activating a center of gravity,
actuating center of gravity.
With SWISH, users, through the headset,
can see a virtual fluid.
As they move the bucket around, they
can feel the center of gravity of the bucket shift
in real time in response to the movements
of the bucket in their hands.
In designing SWISH, we had four design considerations
crisply in our mind.
SWISH should accurately and quickly
respond to the shift in the center of gravity.
The center of gravity that has been
calculated with the software.
SWISH should service different volumes and viscosities
of rendered fluids.
SWISH should be made with off the shelf and easy to reproduce
components so the maker community can easily replicate
it and make it different sizes.
So SWISH, finally, also should service the maker community
to be able to reproduce it in different sizes
for the different situations that they
would like to experiment with virtual fluids.
We achieve the spatial tracking of SWISH
using HTC Vive VR Lighthouses.
Inside the unity, we calculate the center
of gravity of the liquid particles using NVIDIA Flex.
Our SWISH software driver converts the position
of the center of gravity to this vessel center of gravity
and also converts those into motor commands.
Motor commands are being sent via Bluetooth
to the microcontroller.
The microcontroller actuates the motors.
Center of gravity shifts.
At its core, SWISH has a novel three degree of freedom Gantry
system inside.
We actuate the system using four motors.
One motor at the bottom gives us the rotation,
the two on the height give us vertical actuation,
and one gives us the radius, a radial actuation.
At the bottom, we have a homing mechanism using [INAUDIBLE]
sensors and magnets.
And at the top, we have a slip ring
to make sure that the wires are not going to get tangled.
What we demoed yesterday, and I have just shown in here,
is the current version of the system.
Designing this has been fun.
The first iteration of this was quite literally
a bucket, the biggest bucket that we
could find in a hardware store.
We drilled a hole at the bottom, slapped [INAUDIBLE]
at the bottom, put two more motors for the radial actuation
and one for the rotation.
It weighed somewhere around 12 kilograms.
It was noisy, heavy, bulky, and slow, but it kind of worked.
So it was fun and we realized that the shifting of the mass
is a phenomenon that we can explore.
So we went back to the drawing board.
This time we used DC motors.
Instead of controlling them, controlling mechanism of open,
we used close feedback loop system of PID.
And that gave us accurate positioning
of each and every motor and we got to where we are.
We chose six characteristic actions to evaluate SWISH--
sway, swirl, pour, roll, jump, and saute.
These actions were chosen, we chose them
based on our daily interactions with liquids.
For instance, when we have a flask
and we want to know if there is any water inside, we sway that.
We evaluated SWISH based on those actions
and SWISH was able to follow the path of center
of gravity and those locations with minimal latency
and high accuracy.
Analyzing those numbers also showed us
that there are occasions that SWISH deviates from the path.
In this case, you see that in the rotation.
And that's because of the movement
of the force around that axis.
Later on, I'll be discussing our solution to it.
We also evaluated SWISH using a user study.
We asked 24 participants to interact
with fully identical buckets.
The buckets were the same in terms
of size, shape, and weight.
The first bucket had real water in it,
the second one had SWISH mechanism inside,
and the third had solid weight in it, non-moving solid weight.
We tested it using two various viscosities,
simulated viscosities, one lower and one higher.
Then we asked the participants to interact
with each and every bucket and then rate
the buckets based on the realism of the haptic simulation.
Analyzing the results showed us that SWISH in every occasion
beats the solid bucket.
It comes close to the real water bucket in the lower viscosity
simulation.
And in higher viscosity, not only beats the solid bucket,
but comes even closer to the bucket with real water.
And that was kind of expected because water can not
change its viscosity, but SWISH can
adapt to programmable fluids.
In the second user study, we asked the participants
to pour the liquid contained in all of those buckets,
in each and every one of them, into a graduated cylinder
up to a certain point.
Please note that the liquid is not actually leaving.
What they see as liquid leaving but all of those buckets
were contained and sealed.
Analyzing the results showed us that again SWISH
outperforms the solid weight bucket
and comes really close to the bucket containing water.
This actually surprised us because the mass is not
actually changing, so what is happening?
Well, it turns out that our work is actually supported
by studies in psychology.
That moment of inertia and also fatigue
around the wrist of the users is helping us
in creating this illusion of weightlessness.
After this study, we asked the participants
to answer some open-ended questions.
Users found SWISH to be responsive,
to be realistic in providing haptic sensations related
to the experience that they had, to be
as smooth in transitioning from one location to the other,
and also to be adaptive to various viscosities.
Users also reported to some of the shortcomings
of this system.
When we made SWISH, when we chose DC motors
over [INAUDIBLE] motors we knew that the noise was
going to be lower than before.
But still, SWISH was noisy.
And when you are in an environment and all you can see
is the virtual environment, and you don't have any headsets,
those noises become more prominent.
So users found that distracting.
Also, to power up the bucket, we had wires coming into it
and that became distracting for the users as well.
We designed this current version of SWISH
to have a more pronounced mass shifting experience.
That means that the mass inside had to be around 500 grams
and the entire system had to be big enough
to create more moments around the rest of the users.
That made the bucket a little bit too big.
And some of the users found that to be distracting
because in those interactions, they
would have to hold the bucket with two hands.
And that was not very easy.
This motivates us to look into future development
in this work.
We are looking into various ways of dampening the noise
and vibrations associated with the system, for instance,
creating chambers around the motors
or giving noise cancellation to the users.
Also, we are looking into multisensory experience.
Users reported that they would have
preferred to hear the sound of the liquid itself as well.
So we are looking into ways to have
this multisensory experience to make this experience
to be more pronounced.
Also, we are looking into creating multiple vessel sizes
so users can hold the bucket in one hand
and also have multiple versions of it.
You could have a beaker in one hand, augmented with SWISH,
and another one augmented with SWISH as well.
And as you start moving your hand
and pouring liquid from one to another, both of those
would give you the sensation.
We are also looking into ways to make the motors to become
more efficient.
I mentioned before that SWISH, even though successful,
and fast, and accurate, was not able to follow
the path in some occasions.
More importantly, in the rotational axis.
So choosing a motor that is going
to have more torque but with enough of a speed
to give that sensation is one way to go about it.
Also, when we reduce the size of the system itself,
that is going to be helping us as well.
So SWISH is our first step in creating vessels
for virtual environments that can
shift their center of gravity.
Providing haptic sensations in virtual environments
will transform the future of education
and our work is just the first step.
And we are looking forward to have more work in this area.
With SWISH, users not only see a real liquid,
but also feel the liquid.
Thank you so much.
[APPLAUSE]
All right.
We've got lots of time for questions,
so if the student volunteer can come up with the mic.
In addition to the microphone, also passing out
beads to incentivize questions, as is the session next door.
So at the break, if you see people with beads,
you'll know they are positive question-asking members
of the community.
Steven.
OK, so really nice stuff.
One of the issues, of course, is though you
were pouring the water out or pouring the SWISH fluid out,
it didn't really pour out, right?
Which meant that you had a more equal footing
because the water-containing thing actually
weighed the same amount at the end as at the beginning.
So have you thought about how you
would deal with something which really
did appear to lose weight and mass as you poured stuff out
of it?
Thank you.
So there are works in open systems
that you could add mass into a system,
if you need to, in order to change the mass in real time,
and that has been explored.
With a closed system, we have not
found a solution that would give that sustained interaction
or sensation that would feel like,
oh it's completely weightless.
We have not found it yet.
But I think that's the next step of the illusion for us,
to find that sensation that would
create that sense of weightlessness
with a closed system.
None of the moving parts would leave actually
that would give that.
But it's not something that we have found yet.
Yeah, I can think of things with pulleys,
for example, which externally would support stuff and--
That's a great idea.
--or whatever into it or helium.
That would not be a closed standalone thing.
That's a great idea.
Thank you.
Other questions.
So I've got one while you guys think.
There are beads on the line, remember?
You mentioned sensory experiences,
can you comment a little bit about how much you think
the haptic sensation is what people perceive
over something like, you mentioned sound,
but maybe there's also tactile of the liquid splashing
and you get maybe higher frequency haptic feedback,
or even just seeing the amount of liquid in a bucket.
So what role do you think haptic feedback plays
when perceiving the position of water or movement of water?
So there is a difference between how we sense things.
Of course there's a difference in real life
versus virtual environments.
If we close our hands, because of the numerous sensors that we
have in our hands, we have a really close relationship
with touch.
Is that the case for virtual environments or not?
Work suggests that visual dominance is
something that is helping us.
But at the same time, since we are
short on providing these haptic sensations,
creating an approach that has multiple things in it
would emphasize that, would help with that.
Now, which one has more contribution to it?
My guess is that, again, in that order,
haptic would be something that is definitely important,
but I cannot say exactly.
Would it be higher than sound or would be more
important than even visuals?
I can't really answer that.
But to me, for every user that is actually
is going to be different as well.
Because some people-- and that was the case with our users
study as well--
some people mentioned that they actually
wanted the sound so much and the others didn't really
notice it to become something that was distracting.
Sounds good.
We do have time for more questions.
I see a hand.
I'm just wondering, is this the only design
that you thought about, the kind of motorized center of gravity
approach?
I was just wondering, the best simulation
of physical phenomenon is probably
the physical phenomenon.
And then I was wondering could you
build a system where you basically
change the viscosity of the liquid in there
and you could do something with balloons
that self-inflate to offer more resistance or something.
So is that something you experimented with and decided
this is the better approach?
Or yeah, I don't know.
Other thoughts?
Thank you, that's a great question.
So in terms of the mechanical system, then
I'm going to answer your question directly,
but in terms of the mechanical system
we experimented with delta systems,
also with cable driven systems as well.
But in terms of having something,
for instance, some beads of magnetic particles
that then you would magnetize to the site
and then you would have, because of the increased force,
you would have that sensation, that's something
that we're looking into in the future
because if we can create that, then it's going to give even
a more realistic sensation.
We have not found a solution yet,
but that's definitely something that we're looking into.
All right, let's thank Shahab once more.
Thank you.
[APPLAUSE]