Human subject research is arguably the backbone of Human-Computer Interaction
So it is no surprise that researchers spend a lot of time looking for ways to make it easier.
And among these methodologies, remote studies have been especially transformative.
Instead of bringing participants into the lab, you can recruit participants from a much
larger pool, host your study online, get results faster, and most importantly do all this remotely.
Very useful if you find yourself in a global pandemic.
That is, as long as you can find participants that meet the study requirements.
For extended reality research, you often need a headset and controls, a powerful computer to run them on
and a room big enough for lightsaber battles.
Finding participants with access to these is difficult using traditional crowdsourcing techniques.
This contributes to XR research being conducted primarily in lab
Which is a problem if you want to run a collaborative study as you will need several sets of equipment
that will become expensive and impractical quickly.
Or if a global pandemic prevents you from bringing participants into the lab in the first place.
Well, if you can't bring participants to the lab.
I say bring the lab to your participants
And with the help of social VR platforms, that is exactly what we did.
Social VR platforms are distributed synchronous virtual environments where VR communities
can chat, play, and create together.
Most importantly for us, they also often support some level of user-generated content
in the form of custom virtual environments and avatars.
So the concept is simple. Implement your study on a social VR platform, conduct the study
with participants recruited from said platform, analyze the results, and publish your paper.
But is this actually feasible? What types of studies can you do? And will you even get
valid results?
To answer these questions, we conducted two replication studies using the social VR platform
VRChat.
In our first study, we wanted to test quantitative research methods.
So we chose to replicate Fitts Law in VR by Clark et. al.
Our experiment world was implemented using Unity plus the VRChat SDK, and the
Participants were invited to our world and asked to switch to a standardized avatar.
Participants were then instructed to complete a series of selection tasks with targets changing
size and position within the interaction space.
There is no way to connect to external servers from VRChat. So to save results we displayed
the information on a wall, took several screenshots, and used OCR to extract the data from the images.
Our results support the findings of the original work (Fitts law still fits),
and we even found evidence that supports the claim that user experience and better depth cues would lead to faster selection times.
In our second study we wanted to test qualitative research methods, as well as the potential
for social VR to make collaborative studies easier.
We decided that collaborative coupling on tabletop displays from Tang et. al. would
be interesting — even though the original did not use VR, tabletop displays are a common
paradigm in VR applications.
Once again, we implemented the study using Unity and the VRChat SDK
This time we recruited pairs of participants to complete a series of individual and collaborative
tasks using either a filter interaction or a lens interaction.
We recorded these sessions from an overhead view as well as the proctor view.
And then analysed footage to code the participants’ behavior according to the codes described
in the original paper.
We found that the behavior of our VR participants often matched the behavior described in the
original study.
But, notably, our participants preferred different table arrangements and spent less time disengaged
from the task.
Overall, we concluded that many of the concepts established in the original work are transferable
to a VR Context, primarily global perspective sharing is important for collaborative tasks.
So yes, Social VR platforms can be used to conduct remote, synchronous, and collaborative
VR studies.
And, if done correctly, will produce valid results.
If you want a more detailed guide on how we achieved all of this, or want to take a deeper
look at our results, please see our full paper. You can find it and all of our supplemental
material for the studies on OSF.
There is still a lot of work to be done towards establishing remote XR research methods
And it is going to take a community effort to achieve this.
While our approach worked well for this research I would not call it a long-term solution to this problem
but it has the potential to become one.
What needs to come next is either official support for research studies from existing social VR platforms,
or a bespoke platform dedicated to remote XR studies.
Regardless of which path we take, I believe the future of remote XR studies will be
a lot more social.