Hello everyone, my name is Mengyu Chen and I am going to present our work EntangleVR,
which is a visual programming interface that helps users to easily create interactive virtual
reality scenes.
In this work, we look at how the concept of entanglement, a quantum physics phenomena
can be used in application of virtual reality scene creation and how we developed a visual
programming interface allowing people to quickly and easily learn this entanglement idea while
creating their own interactive VR scenes.
If you are not familiar with quantum entanglement, it is basically a phenomenon that describes
a correlated relationship between the measurement of multiple spatially separated particles.
It is a very unique feature of quantum computing that it can transform individual object states
into non-separable shared states. It is quite widely used in quantum computing algorithms
these days because this phenomena of entanglement can create a set of correlated switches and
allow you to manipulate a group of states from a single interaction on one of the entangled
unit.
The image here illustrates an example of entanglement that we can observe a particle at one location
while having another particle affected at somewhere else. So no matter which spin direction
is observed on the left particle, the right particle will always be observed in the opposite
direction simultaneously.
We thought this idea can be very interesting and useful for virtual reality scene design.
Because in a vr environment, a user can interact with many different objects in the scene and
if these objects are entangled, then whenever a user does something to one object, another
object somewhere else in the environment can get affected. You can have many such entangled
pairs of objects and the final rendering of the environment can have various different
combinations and possibilities. So instead of giving the user a static virtual environment
that is pre-determined by the artist, we can add entangled elements into the scene, and
the user’s interaction can become a key factor in affecting the final scene generation,
so that every user has their own unique experience of the environment. This allows for creating
highly dynamic and complex spaces for discovery and exploration. Potentially, we can make
it even more complex and real-world like and have multiple chained effects in the virtual
scene. Our goal was to design a scene full of interactive and correlated elements that
can help the creator build up a story with\h interesting changing dynamics.
Regarding existing methods for interactive VR scene generation, there are some existing
tools that can be used and many of you are likely familiar with. However, most prior
research and commercial software on the market focuses on composing static VR scenes or scenes
with animated graphics but there is no user interaction involved during scene generation.
If the user wants to make an interactive VR scene, then there are only a few options available
using professional game engine pipelines such as Unity or Unreal, or programming API’s
such as A-Frame. But all of them require the creator or the designer to have good bit of
programming experience\h even for creating very simple interactions.\hand it takes quite
a lot time and effort to learn how to use these tools. Our work is inspired by a few
prior researches such as Takala’s RUIS VR prototyping system in 2014, Nebling et al’s
interactive AR prototyping system in 2019, and Zhang et al’s immersive and reactive
vr programming tool in 2019. However, we are focusing on introducing the idea of quantum
entanglement as a new way to bridge user interaction and vr scene creation along with an interactive
creation process for the creators and designers of no or very little programming knowledge.
Our system EntangleVR provides a reactive visual programming interface that lets creators
build virtual environments using the entanglement concept without requiring any prior knowledge
in quantum mechanics. We also designed a set of quantum computing inspired operations,
interactions and visual representations for virtual object behaviors. Particularly our
system lets user participate in the generation of virtual environments by their interactions
and makes it possible for the user to influence the final outcome of the VR scene based on
probabilities and entangled relationships. Our system also helps user learn about some
basic quantum computing related concepts.
The video clip here briefly shows the creation process and the final outcome when the user
is in VR mode interacting and affecting the entangled virtual environment.
EntangleVR is built as a Unity plugin with a node based graph interface and a custom-built
quantum simulator running in the backend to simulate the quantum-like behaviors of virtual
objects. The main user interface has four components.
The tool box has buttons to create various types of nodes. We will introduce what these
nodes are in the next slide.
The graph canvas here is where we can manipulate and link different nodes to formulate a graph.
It is same reactive as the nodes and will take effect immediately once the nodes are
connected.
There is also an observation area that has an interactive preview of the scene composed.
The creator can play with the scene elements directly in this mini playground with a mini
2D avatar and can test how the interactive components works.
We also have a tutorial area that walks users through the system along with guided tutorials
to help the user interactively learn how to use the system.
As our system is inspired by quantum computing, we want to help the user gain some sense of
quantum computing first. We built this slowly from basics, starting with the notion of a
qubit which can then\h help them better understand how entanglement works. We have qubit node
that is visualized by a bloch sphere and can be initialized to several preset states. We
also have a quantum gate node, that allows creators to manipulate and control a qubit’s
state. The system introduces a new type of\h virtual object called Super Object. It basically
lets you define a two-state virtual object that simulates the uncertain super-position
nature of a qubit. So a super object by default has 50% chance to be measured at state A,
and 50 % chance to be in state B. We use a qubit’s probability amplitudes of 0 and
1 to control a super object’s probabilities of the two states. In the example image here,
a super object can be either an apple or a banana. If the input qubit has a 100% chance
to be observed at 0, then the super object will then have a 100% chance to be observed
as an apple.\h
Here is an animated version showing how these nodes help the user easily map the probability
to the super object with visual references and control its properties.
We also provide an observer node that provides an interaction method to measure and collapse
a super object. When a super object is created, it does not have a definite state. The user
can choose to observe the super object and turn it into classical state either state
A or state B. The observer node here tells you the probabilities of the linked super
object, so that the creator can have a better sense of what it may turn out to be. We also
map this observation action into VR mode as touching gesture. That as soon as the user
touches the super object, an observation action will be triggered and the super object will
collapse into a definite state based on its input qubit’s probability amplitudes.\h
Then we have the entangler that we can use it to set two super objects to have correlated
states during observation.
We defined two types of entanglement in our current system: same and opposite.
For the same mode, when either one of the entangled super objects is observed, the other
super object will also get affected and turn out to be in the same state as the first observed
super object. Slightly different from the actual quantum mechanics, here are determining
the final outcomes of these two super objects purely based upon user’s interaction preference.
In the illustration here, if the user prefers to interact with the top super object which
has a 50:50 chance to be either state A or state B, then the final outcome will be either
state A + state A or state B + state B. If the user prefers to interact with the bottom
super object that has 100% chance to be in state A, then only one possible outcome can
be observed for both of the super objects, which is state A + state A.
We designed such an entangled play for easier understanding and for a goal to give user
interaction a more important role in determining the final outcome of these entangled pair
of virtual objects.
The opposite mode works in a similar way but has the opposite effect.
Because we are using the qubit’s probability amplitudes to determine the observed results
of super objects, whenever we create a super object we are creating two possible outcomes.
Having two super objects means we have total of 4 possibilities. We also have a feature
letting user to create super location so that we can have more combinations from object
states and their other properties. But even with this many combinations that we can get,
we are still controlling all the probabilities individually via different gates, so the entire
scene is still controllable to some level and the outcomes are likely predictable.\h
Another possible benefit of this type of probabilistic control via the qubit’s probability amplitudes
is that we can change the computational basis of measurement and change how we want to do
the observation, and if we do so, the probabilitiy distributions in the entire virtual environment
can be shifted together to fit the new computational basis, adding more space for creative play
and control.
Here is a visual outcome from our interface usability study that we asked participants
to independently create a mini virtual environment with entangled objects. The apple and the
flower in this scene cannot appear at the same time based on their entangled relationships.
The relationship rules are when the virtual human character touches the apple, a zombie
appears and destroys the flower and when the flower is touched, a deer appears and eats
the apple.\h
Here are all we have for today. If you are interested in knowing more details, please
checkout our paper. Thank you!