This video is a collaboration with École de technologie supérieure, its student
association and the CHUM research center.
You probably know this optical illusion...
It looks like the arrows are not the same length when they
actually are.
The very first time that I was exposed to this kind of illusion, I got tricked
just like everyone else, and I was really disappointed with my brain.
It seemed to me that it should have figured it out.
Since then, uh-uh, I've learned my lesson and it doesn't happen anymore, I know exactly in
which contexts I cannot trust my senses, so that I can be even more rational.
So I am kind of convinced that I am capable of being in control of what I see,
what I experience, what I feel... Or I should say that I was convinced...
until I had the virtual reality experience that you are about to see.
Let's say that it turned my certainties upside down... and if you are convinced that you are still
in control... that's cute.
We shot these experiences last summer, and since then, Facebook has announced that they
are working on the “metaverse”.
You know, it's the idea that the future of social networks and digital life
would take place in a 3D virtual environment (aka virtual reality).
This may taint my conclusions in a way that I had not anticipated when
we shot this... But back then, when we were shooting these experiments, Facebook's Metaverse
had not been announced, I was still convinced to be in control of what I see and
what I experience, and by golly, we were having a good time.
You're about to enter the virtual world.
We are at a virtual reality lab used for medical research.
What you see there are cameras which capture in real time the position of
all the little white balls that have been placed all over my body.
From the position of each of these balls and my measurements: my height,
neck circumference, chest, waist, etc, which we will keep confidential, the software is
able to create my digital double, my avatar.
All you have to do is put this double in a virtual world where each of my movements
will be transcribed.
3-2-1… Ok.
There's even my shadow! Oh! That's fun.
Experiment 1.
Careful, now I'm gonna put you in the city environment.
Oh yeah… This is where you also see the power...
so this is a feeling of presence, the fact that I think that the virtual
environment is real.
The graphics are not super HD, but they're enough for your brain to be confused.
Yeah really, I know there is no issue on the side but I don't want to go there.
I am ridiculous.
I know I am being ridiculous.
Dare I...? (walk into the void)
*taps the ground with her foot before going*
I had to touch, I had to get some sensory information to tell me
that it was okay.
Okay well here is good, too.
So let's be clear.
I know for a fact that I am in a lab and not on top of a roof over a city.
So you might think I'm exaggerating my reaction and that I'm pretending
to be uncomfortable.
No, there would be no point in doing that.
In fact, the best proof I can give you to show that I am really disturbed by
what I see is this.
I lose my balance.
But the loss of balance is also related to the fact that here, I can see that
there is a ledge and I know I can't go backwards, even though I can!
Yes, technically you can... I didn't think I would get so...
Carried away by the environment? Yeah, because it’s easy now.
*walking along the board without the VR helmet* That was the first thing that Valentin
wanted me to realize: presence.
Even though I am perfectly aware that I'm in the lab...
There is a part that I cannot control.
My brain reacts to the virtual environment it's in.
I don't dare to walk off the plank, I lose my balance... My reflexes follow
what I perceive and not what I know to be true.
Which is ultimately quite logical; in an emergency when you lose your balance,
it makes sense to trust our senses first rather than rationality.
So at that point I still think that while sure, my brain has lost its bearings,
I am still in control of what I do and how I move.
Until the second experience...
Experiment 2.
Now you're going to point with your left hand where you think that
your right hand is.
I did what I was told and pointed at my hand.
More precisely, I was trying to point at the base of my fingers, in the center.
That was the first try.
Easy.
What I saw in my helmet represented reality.
So I had no problem spotting my hand.
But Valentin had explained to me the objective of the experiment and I knew
what we were going to do next.
Which makes me, by the way, a terrible experiment subject.
I knew that he was going to shift my virtual hand by turning it little by little, subtly,
so that what I see in my headset no longer corresponds to what I am actually experiencing.
So my virtual hand was going to be shifted further than my real hand.
Since I knew that, I tried to be clever and to show off that I could
very well distinguish reality from the virtual world, totally convinced that I would manage... So when
I could tell that he was going to his computer to change the parameters of my hand,
I knew that was when he was going to shift my hand virtually....
which of course led to this reaction:
I feel that my right hand has been shifted to the right.
I feel it here and I see it there.
OK.
The second time when you told me that you thought it had been shifted, it
actually wasn't... I tricked you by pretending to go like...
I'm typing on the computer and stuff.
I feel that my right hand has been shifted to the right.
OK...
The joke's on me.
I thought that my hand had been shifted when it wasn't...
This goes to show how easily, despite our best efforts to be rational, we can still get
tricked... And yet, I had all my senses on alert to pick up on it...
Right now, I feel super aware… “aware”, I feel like Jean-Claude
Van Damme... I feel like I am particularly attentive to the feeling
of my hands on the table.
Which I wasn't before.
In fact, my brain did a kind of mashup of all the information at its
disposal: the virtual environment, my preconceptions (what I thought was going
to happen), my proprioception, tactile and auditory info... all that to give an answer that happens to be
not quite right.
We continued the experience by shifting my virtual hand little by little,
for real this time.
But I still tried to beat the system...
I see it there... I'm figuring it out based on the rest of my body.
That's normal.
If I am being a smartass, I think it's there.
In any case your brain makes some kind of a combination of all the stimuli it
receives: your proprioception has helped you a lot but it isn't that precise
actually, and then you have visual, auditory, and tactile information.
Your brain ends up combining everything to reach its conclusion: "ok, well I think
that my hand is there”.
And when you told me... visually you said "it's there, but I think it is
actually here”, and you pointed closer to your hand, but you didn't point directly
to it, in fact you were in an in-between spot.
This is the pot pourri I was talking about: my brain has averaged all the
stimuli and despite my best efforts, I could not ignore the virtual environment.
For comparison, if I do the same exercise with my eyes closed (which you
can do too), I do it pretty well, this is where I was pointing.
So it's not that I'm naturally bad at this, or that I have poor proprioception,
this is really coming from the virtual environment.
The fact that I couldn't really locate my hand in this context, we call that
the proprioceptive drift.
Proprioceptive drift is only possible because I have associated this virtual self
to my real self.
I embodied, that's what it's called, this virtual character.
This means that my brain recognizes that the arm in front of me, even though I know
rationally that it is not my own because it is virtual, I still understand
that it's mine... virtually.
We must create multisensory stimuli, with visual and tactile stimuli,
for example, to be linked.
For example, if you see a rubber hand, a virtual hand that you see in virtual reality
which is touched exactly at the same spot as your real hand. Your brain will
think that... well that means that this is my own hand if I can feel it there.
And if it makes perfect sense to your brain to consider a rubber hand as
being your own hand, in that case you will enter into a state of embodiment towards this
rubber hand.
Even though we can see that it is a rubber hand…?
Even though we can see that it is a rubber hand.
Valentin was smart to trick me in order to demonstrate to myself that I was not
that clever.
At the time, a little seed had been planted in my brain that this virtual environment
could alter my perceptions.
My certainties took a hit then... I don't know how well you can
perceive what this implies without experiencing it yourself, so the third
experiment may help you out...
Experiment 3.
Here, Valentin had placed me in front of a virtual mirror that reflected my avatar.
So my brain realized that she was me.
I started experimenting, testing the limits of the thing.
Fortunately, ridicule doesn't kill… Once I made that visual link, with
a motor link too since my movements were connected, Valentin added
a third tactile stimulus by touching me with a stick that I could also see
in my virtual world.
The point of that is that the more stimuli we add from different sources,
the better this phenomenon of embodiment that we are looking for is likely to work.
After the visual, motor and tactile stimuli, my brain was all set for this final experience.
Ok, let's start the last one, so close your eyes... You can open your eyes.
Valentin has artificially changed the length of one of my legs.
What is interesting and what differentiates this kind of experience from previous ones,
is that we are trying to quantify how much I am influenced by
virtual reality.
Under my feet there are pressure sensors which measure how I distribute my weight
between my two feet.
All we have to do is compare how these measures change between the avatar that looks like me
and this fake avatar who has a longer leg.
That way, we will be able to quantify the influence of virtual reality
on my movements.
The result?... This is the frustrating part of the video because it's too early to tell.
Now, we're getting into current research since this experiment is the one that Valentin
applies to his participants as part of his PhD research.
But! The point that I wanted to bring you is already in the scientific literature.
For example, in a study where participants embodied a Godzilla which has smaller
legs than we do, the participants started to take on a smaller step amplitude
with a higher frequency... as if they actually had smaller legs too.
So has their gait been changed because of their preconception of how
Godzilla moves in movies, because of the effect of the smaller legs visual,
or both? This remains an open question.
And this is the frontier of our knowledge on virtual reality: we see that we
may induce a motor change in a person by getting them to embody a character other than themselves.
I repeat: we can induce physical changes in a person when they are embodied
in a character other than themselves.
But it's not yet fully characterized, we are at a crossroads.
So.
If we manage to cause a motor change in healthy people who are embodied
in a Godzilla body, then we can probably also induce it in patients
to correct their posture, the way they distribute their weight between their
two legs… etc.
And that’s the point of Valentin’s experience.
Clinical applications for people who are hemiparetic, for example,
say if someone has had a stroke, they have a hard time moving one side of the body more than the other.
And in this case, we could develop rehabilitation exercises with a personalized avatar
who will move further than what the person is doing.
And the person will tend to follow this avatar.
So they will end up having a bigger range of motion, shorten rehabilitation time, things like that.
This is a future use case that we could imagine.
When you think about it, our brain is used to these avatar changes.
We go from baby, to child, to teenager, to young adult to an adult that's less and less
young… Our self-perception changes over time and our brains know how to adapt.
With virtual reality, we get to exploit this flexibility, this plasticity of the brain...
which is not applicable only to our motor function, by the way.
There are other aspects.
This can also be applied to a psychological level.
Some articles show how our behavior changes depending on the size
of our avatar (we become more or less confident) and even when changing the color
of our skin, and we can also feel fatter or thinner depending
on what we see.
And even further, it can be used in psychiatry for patients who
have schizophrenia.
I hope these few examples will pique your curiosity because we
won't talk about it now, but we will talk about it tomorrow, live.
But before I give you all the details, I would like to dwell for a moment on the
potential flip side of this kind of technology... and here we enter the
personal opinion section.
Facebook have recently announced that they were working on the metaverse, a digital future,
which is not super clear yet, but which would take place in virtual reality
and/or augmented reality.
I am absolutely certain that there will be great aspects to this kind of technology.
But, this social network is one of the main platforms that spreads disinformation,
and doesn't have a very strong ethical code, if I may put it very mildly.
What I've learned from this video, is just how vulnerable we are
and how easy to manipulate we are (with or without virtual reality).
So my hope is that by the time this technology is in place and democratized,
that it is regulated and verified by independent external parties, so that
we don't have to rely on the good faith of a conglomerate (Facebook or other),
so that we may keep the awesome parts of virtual reality.
The coin without its other side...
Utopian?
Thanks for watching this video, I hope
that I made you want to know more because I invite you to
meet up for a livestream tomorrow on Twitch.
All the info is in the description.
We will get into some elements that we have briefly mentioned in this video, especially
the multisensory aspects but also what I quickly mentioned about
skin color, schizophrenia and self-image, all of that.
But of course, we will also answer your questions.
Ask them in a comment under this video with the hashtag #QLive.
These are the questions we will focus on during the livestream, although we will also take
some questions during.
See you tomorrow?
This video is a collaboration with the
institutions with which Valentin is affiliated: the École de technologie supérieure (ETS),
the CHUM Research Center which is where the Laboratoire de recherche en Imagerie
et Orthopédie (LIO) is located, and the ETS student association.
Thanks to them.