Hi, my name is Elodie BOUZBIB, I’m a PhD student at Sorbonne Université, and I’m
about to introduce CoVR, a novel robotic interface providing strong kinaesthetic feedback in
a room scale VR arena, that we developed with my supervisors Gilles Bailly, Sinan Haliyo
and Pascal Frey.
A main challenge in VR is the lack of convincing haptic feedback.
While VR aims to create more and more natural interactions, it is still difficult to be
able to replicate it entirely.
For instance, how to realistically touch a wall?
The user would touch it, push on it, lean on it, and eventually redirect herself towards
an exit door.
But today, in VR, it is common to use controllers.
They may vibrate when we virtually touch a wall but they do not constrain the user as
they focus on the users hands, and as the user would expect force-feedback on her whole-body,
eventually she is more likely to go through the wall.
This presentation aims at demonstrating that virtual reality can benefit from large-scale
force-feedback and shows an interface that enables it.
One of CoVR’s main features is large-scale force-feedback.
CoVR has a sufficient robustness for users to actually lie on it or even push hard on
it.
Moreover, CoVR can reach any location of the VR arena, Here it actually displaces itself
to overlay these three different doors, without knowing where the user is about to interact.
CoVR is a XY Cartesian robot, moving over two axis like an XY plotter with pulley-belt
systems.
It is mounted over the ceiling, and it displaces a physical column to overlay physical objects
over a large VR arena.
So now, how to interact with CoVR?
When no interaction is required, CoVR remains out of reach and users can wander in the whole
arena.
CoVR does not constrain the users to hold continuously a controller and actually encounters
the user when an interaction is required.
As we’re interested in interactions, let’s talk about the available interactions whenever
CoVR is static, which means it doesn’t move unless it is for reaching an object of interest
position.
The force transmitted is this configuration is from the user to CoVR.
On CoVR’s panels, we can create some haptic rendering to overlay objects or even suggest
shapes or codes to the users.
For instance, here the user is exploring a code with her bare-hands, as her vision is
blurred.
We can also attach props to CoVR’s panels, for users to perform a "mechanical manipulation"
or "directed manipulation" to actuate props as per this slot machine.
CoVR can transport objects for the users to manipulate.
She can freely manipulate a cube to fit it in a locker.
But here, the cube is actually too big to fit in the locker, and she will feel it as
CoVR enables both free manipulation of untethered objects and interactions between objects.
Whenever CoVR is static, it allows both bare-hands exploration and manipulation, with over a
100 Newtons perceived stiffness.
It also allows multiple postures while remaining static.
As previously seen, CoVR enables the users to push hard on it, and other postures are
available, such as sitting comfortably on a chair, going into CoVR to reach a higher
target or even going underneath CoVR when playing an obstacle course game.
With its robustness, CoVR also allows users to lean on it.
Users can also receive interactions from CoVR whenever it is dynamic.
In this configuration, CoVR transmits the force to the user.
CoVR’s key feature is to enable whole-body interactions with large kinesthetic feedback.
These are even stronger whenever we take advantage of CoVR’s displacements.
For instance, users can be led through forces and pulled by CoVR, as if they were travelling
with a magic broom.
They also can be literally transported by CoVR within the arena, as it can carry over
80kg of load capacity.
CoVR can also initiate the interactions, and simulate the crossing of a ghost through the
user’s head.
Whenever CoVR is dynamic, it provides large-scale force-feedback, and novel interactions such
as transport by sitting on CoVR or pulled through traction forces.
For all interactions to happen, the main concern is to physically and accurately match the
Virtual object of interest positions prior to the user.
But how to ensure that CoVR is at the right position at the right time?
How to decide what to overlay?
Let’s take a top view of our arena.
The implementation is done on Unity, and here the avatar is at the center of the arena.
If we add an object of interest, it will be straight forward for the user to reach it.
Now let’s add CoVR, at the bottom right of the arena.
What trajectory should CoVR follow?
The shortest trajectory from CoVR’s origin to an object of interest would collide with
the user.
We then decided to add a cone-shaped virtual obstacle to the user on Unity for CoVR to
avoid it smoothly.
To enable this, we attached a proxy to CoVR, being a ball subject to gravity on Unity.
This proxy ball will hence respect the physics of the game engine and lead CoVR around the
user and its cone-shaped obstacle.
Now we need to give a command position to CoVR.
Let’s imagine a non-deterministic scenario, where the users are free to interact with
any object of interest.
Non-deterministic scenarios introduce uncertainty regarding the object CoVR will overlay.
In order to be available prior to interaction, we need to give a command position to CoVR.
Naturally, CoVR will firstly move towards the objects of interest barycentre, the objects’
centroid, to be equally available to overlay all of these objects.
We can see CoVR here placing itself at the barycentre of all the objects of interest.
The idea is for CoVR to move accordingly with the users’ displacements and more precisely,
her intentions.
Whenever the user is moving, her intentions are changing.
To predict which object of interest is about to be interacted with, we calculated a weight
per object of interest, as a function of the users’ orientation and distance to them.
CoVR hence goes at the weighted average of the objects of interest positions.
When a weight is equal to 1, the object has been chosen.
We can see here that the proxy, leading CoVR, is moving towards the object that has been
chosen by the intention algorithm.
The weight depends on the Distance of the user from the object as well as its orientation.
The relative contribution of the distance and the orientation depends on an omega factor.
It is not clear what the optimal value for omega is.
Should the distance or orientation be more important to predict the next object of interest?
We thus conducted a technical evaluation to estimate the optimal value of omega.
The optimal omega will allow to predict correctly which object is about to be interacted with.
We also want to evaluate CoVR’s accuracy depending on the number of virtual objects
of interest within the scene, and to validate that CoVR physically follows its proxy and
hence that CoVR is safe around the users.
We performed a data collection of 6 users, that were asked to perform an explorative
task and touch target balls at random positions and orientations among distractors.
We control the number of distractors: from 0 to 4, and they appear in a pseudo-random
order.
All participants performed 10 blocks of 5 conditions.
We recorded a total of 300 trials, some trials having a single target object, no distractor.
Some trials having multiple distractors.
We recorded the user location and orientation, and the objects of interest’s positions.
Then we had a simulation phase, where we used user’s data to physically run CoVR in the
arena.
In the data collection, we had REAL users but NO column.
In the simulation, we use users’ data but we use a REAL column.
We then perform a simulation in an omega fitting phase, where we changed Omega and measured
the success rate of CoVR reaching a VOI before the user, the distance between CoVR and the
target when a collision occurred.
We first performed a broad exploration of omega, then refined it to find the optimal
one for each condition.
Here for instance we can see the user moving, the weights changing accordingly with it,
and the proxy and hence CoVR are moving according to these weights.
The average accuracy between the carriage and the target centres was 1.8cm over 300
trials.
We also measured the distance between the proxy and the carriage centres.
The average was below 1cm, which ensures they share the same trajectory.
We found that with omega at 0.175, we had a little over 80% success rate.
The orientation to an object of interest is hence way more prominent than the distance
to it.
Let’s think of a way to increase this success rate.
We can picture a basketball game.
Players will most certainly interact with the ball prior to interacting with the hoop
right?
We can hence assume that the ball is more likely to be interacted with.
If we translate it to CoVR, CoVR providing a ball first would hence make more sense than
CoVR being at the hoop position.
We can then add "prior probabilities" on what we call a "scenario-based model", which is
still non-deterministic as it doesn’t force a position on CoVR and still depends on the
users intentions.
We launched a second simulation by adding 75% prior probabilities on the target to be
interacted with.
This improved the prediction with an average success rate higher than 95% over the 300
trials.
Now that we defined that CoVR allows non-deterministic scenarios and whole-body interactions, we
conducted a user experience to validate our implementation and verify that CoVR does not
collide with the users in real life experiments.
We also wanted to evaluate the strength of our force-related interactions on users.
Another goal of this experiment is to evaluate if large force-feedback is appreciated in
VR and what were the users favourite interactions with CoVR.
The participants were asked to climb on a stair, as seen previously.
Then they had to push as hard as they could on magic walls.
They were then transported on a magic broom and pulled through forces, crossed by a ghost
and physically transported in the arena.
No collisions with the users or mismatches regarding their intentions was noticed during
the games.
This validates our safety and software implementation.
Users spontaneously pushed hard on the walls and ranked their own forces with a 5.5 grade
on a 7 point Likert-scale.
The force received by CoVR while being transported was ranked with a 6.1 on a 7 point Likert-scale.
This validates our mechanical implementation concerning CoVR’s robustness.
In terms of interactions, the global enjoyment was above 6 over a 7 point Likert-scale.
The favourite interactions were being pulled through forces, with 37% of the votes, while
the favourite one was the transport, with 63% of the votes.
We can hence affirm that VR can benefit from large force-feedback and that there is a promising
future for whole-body interactions with large force-feedback.
To sum it up, we presented CoVR, an Interface allowing whole-body interactions and strong
kinesthetic feedback in a room-scale arena.
We also presented the Elaboration and Evaluation of a user-intention based model allowing non-deterministic
scenarios, and Empirical results on the users most enjoyable interactions using CoVR.
Thank you for watching this presentation.