Hi everyone,
my name is Tim Zindulka.
Thank you for your interest in our research in performance and experience of throwing in virtual reality.
This is joint work with Miroslav Bachinski and Jörg Müller.
Virtual reality is very popular with many different areas of application.
Throwing, in both the virtual and the real world, is a fundamental movement.
Given this, we could expect decent throwing mechanics in virtual reality.
But oftentimes, even when trying to hit a close trash bin…
…or close targets, the throws fail miserably.
So despite the ubiquity of throwing, accurate throwing in VR is surprisingly difficult.
For this research, we tried to answer three basic questions:
First, is throwing in virtual reality worse than in the real world?
If it‘s more prone to errors, why is that?
And last, how could developers improve the throwing performance and experience in VR.
The cornerstones of this work are of course literature from classical physics and sports science.
To the best of our knowledge, there is no research or system that deals explicitly with the basic problem of throwing in virtual reality.
Nevertheless we could draw a lot if valuable information from previous publications that consider specific VR applications and sometimes even investigated the throwing mechanics involved in those applications
These include combinations of throwing aided by digital cameras, Kinect Sensors or mobile devices.
Several works investigate training and its transferability between VR and real life.
For example looking at basketball free-throws or darts.
Others propose new ways to throw in a virtual environment or look at very specific problems such spatial distortion in VR or early kinematic throwing cues.
Before we‘re getting into the cool stuff, we want to have a look at some basic physics to facilitate a better understanding for some concepts occurring later in this presentation.
When a body is thrown at an angle with an initial velocity and the only external force along the trajectory is from gravity, it moves in a bilaterally symmetrical, parabolic path.
According to the superposition principle the initial velocity is set by the separate velocities along the horizontal and vertical axes.
Neglecting air resistance, the horizontal acceleration is constant on both the horizontal axis, where it equates to 0 and the vertical axis where equates to the negative value of the gravitational acceleration.
Therefore the body moves with constant horizontal and linearly varying vertical velocity.
During the throwing motion, the human hand moves in a circular path.
The shoulder joint acts as the main rotation anchor.
In addition, rotations occur around the elbow and wrist.
All we need to understand here is that its important to consider the speed and angular velocity separately, since two bodies with different distances from the axis of rotation at the same angular velocity cover different lengths of the circular paths.
As a result, the linear velocities are also different.
Since our experimental setup minimized the effects of other external factors such as drag and wind, we decided to neglect them and have everything else handles by the physics engine.
On the same note we should mention that we followed the principle of “physical plausibility”,
meaning that we were more interested in what the user thinks is right and not necessarily on what's the most physically accurate.
For our experiment we created a virtual world using the Unity Game Engine, which uses NVIDIA Physx for all physical calculations.
To enable our participants to emerge into VR we used the HTC Vive Pro.
Our experiment software is based on a state-of-the-art implementation of throwing mechanics from the SteamVR Unity Plugin.
The supplied SteamVR Interaction System offers different “Release Velocity Styles”:
We use the default GetFromHand option since it‘s widely used, obtained the best results in a short technical pre-study and is the simplest and easiest to replicate of all the implemented options.
Here, the velocities of the controller are transferred directly to the Rigidbody of the thrown object.
This simple yet realistic method establishes a baseline for VR throwing that will remain valid in the future.
Thus research on improved throwing algorithms can compare itself against this technique.
So, when the user releases the Trigger-Button by 30% (that a default value), the virtual object is released from the hand and the physics engine takes over.
Upon pickup the object snaps to the center of mass of the VR-Controller.
(Which is marked in red here)
This adjustment was necessary because as we just saw in on the physics slide: the tangential velocity depends linearly on the radius.
Meaning if the object could be attached to the controller at a fixed position, the distance from the object to the wrist (so from here to here, the rotational anchor) might be greater and thus the object would be catapulted with a much higher or lower velocity than intended.
We conducted a pre-study to explore the effects of learning and fatigue on throwing in VR.
This enabled us to determine how many repetitions were needed for the main user study.
For this we recruited 3, right handed, male participants.
They had to perform a series of overarm throws at a target placed in close proximity.
We instructed the participants to hit the target center whenever possible.
First participants completed 10 real throws and then they were instructed to throw a total of 200 times in VR.
The target and the distances were exactly the same for both environments.
For the results:
All subjects showed symptoms of fatigue after 50 to 100 throws.
One subject even stopped after half the throws because of shoulder pain.
The figure illustrates the accuracy of each hit point for the three participants over the course of the session.
Without going into too much detail here:
The biggest takeaways are:
First, there were no improvements in the accuracy after some initial 10-15 throws.
Second, after about 100 throws accuracy decreased due to fatigue, as observed from the participants’ reactions.
In conclusion we set the number of repetitions for the main user study to 20, so the throws will take place where throwing stability is increased, and the influence of fatigue is still very small.
The core of this work is a user study in which we compare throwing in the real world with throwing in a virtual environment.
To cover a broad variety of throwing scenarios we included three different throwing styles.
The first technique is the overhand throw or as we call it the darts-like-throw.
For station 2 participants were instructed to use the underhand-throw with the target facing the ceiling.
The last station we return to overhand throwing but this time at a much greater distance resulting in a very different throwing style.
So, twelve right-handed participants (we had 8 males and 4 females) with an average age of 26 years participated in the experiment.
Five of them had never used a VR-system before.
First and foremost we collected data about accuracy and precision.
Again accuracy was measured as the distance of the impact of the projectile to the target center.
And precision was measured as the standard deviation of the hit points.
Or to make it easier to understand:
Accuracy indicates how close the impacts are to the target center and precision shows how close the impacts are to each other.
We examined kinematic patterns of the hand and of the ball using motion-capture and logging in VR.
Effort was assessed by using the Raw NASA-TLX questionnaire after each condition.
Additional qualitative data was collected by questionnaires and spoken feedback in the end.
For the real part of the experiment twelve OptiTrack cameras detected the movement of reflectors attached to the participant’s hand and the thrown object with 240 frames per second.
For the virtual part of the experiment we used the HTC Pro operating at 90 Hz.
The experiment started with a questionnaire covering demographic information and their previous experience with throwing and VR, followed by a quick introduction.
Participants were then instructed to perform a series of throws in the real and the virtual world, while always trying to hit the target center.
For VR there was a short warm-up session.
As throwing is such a natural movement, we did not have a warm-up for the real throws.
The experimental session consisted of 20 throws for each station.
Other than the general throwing style no instruction was given regarding on how to perform the throwing motion.
The experiment concluded with a second questionnaire covering the qualitative data explained together with the results in the following slides.
We compared accuracy and precision data between conditions individually for each throwing style.
For analysis of accuracy we have computed offsets between each hit point and the center of the corresponding target, and then averaged them for each participant.
For analysis of precision we computed the standard deviation of the hit points capturing their spread on the target surface, where higher standard deviation indicates lower precision.
Throws in the real world were more accurate and precise than in the virtual world regardless of the throwing style.
A Wilcoxon Signed-Rank Test indicated that the measured accuracy in the real condition was significantly higher than in the VR conditions.
The measured distance to the target center was almost twice as high for the virtual throws.
Similarly, the precision was significantly highER in the real condition than in the VR condition again for all throwing styles.
The scatter plots of the hit point distribution with associated 2D kernel density estimator highlight, that while the scattering of hit points is generally higher in all VR throwing styles, the hit points are also much more scattered along the vertical axis.
This indicates that it was more difficult for participants to time the release of the object and adjust throwing power.
We assessed Task Load with the Raw NASA-TLX.
Although we found no statistically significant, we can observe a trend, that the workload was perceived to be higher in VR than for the real setting.
We collected the position and rotation of the ball and the hand during the throwing motion.
The saved datapoints could be viewed directly in the virtual environment to allow for immediate qualitative analysis as seen in the video here.
In accordance with other literature we observed different throwing patterns when comparing reality with VR.
We noted several interesting differences, but the biggest takeaway here is that the throwing motions are indeed not identical.
This is valuable information because we now know that we should try reduce the differences to improve transferability between the two worlds.
Here we summed up the most important qualitative information we collected during the experiment with questionnaires, observations and spoken feedback.
Participants assessed their distance to the target in the real and in the virtual environment similar on average.
They perceived their throwing power, accuracy and precision to be worse in the virtual world.
The controller and especially the button to release the virtual object were perceived to have the biggest negative influence on the outcome of the throw.
When holding a virtual object the brain does not get accurate information about the weight and center of mass of the object.
In our study we let the virtual object snap to the controller's center of mass upon pickup automatically.
None of the participants mentioned the center of mass indicating that this adjustment was indeed very effective.
Now for a quick discussion:
In our user study, we found that throwing in reality is almost twice as accurate and 2-3 times more precise than throwing in VR.
While we can not determine the sole reasons for this inaccuracy through our study, the observations and questionnaires give some possible explanations.
The main complain was timing problems when releasing the virtual object.
This would be consistent with the inaccuracy accumulating in the vertical and longitudinal dimensions.
On future controllers, buttons for the release of thrown objects should allow as accurate timing as possible, for example, by providing a short travel and feedback.
In the meantime, we should expect the user to throw with less accuracy in VR.
This could be counterbalanced by using larger targets, particularly in the distance and height dimensions.
We found that it is very important to align the center of mass of the virtual object and the controller.
This can be done by letting the virtual object snap to the controller's center of mass.
Now in conclusion: we have evaluated throwing accuracy and experience in VR.
We found that throwing in VR is only half as accurate and only a third as precise as in the real world.
This can be attributed to inaccuracies in the vertical and longitudinal dimensions.
One possible cause could be timing inaccuracies in the release of the object.
We provide recommendations to improve on, or mediate, these difficulties in throwing.
Finally, this research establishes a baseline and we hope to enable future research in this way.
Thank you very much!