Hello, everyone. I'm Lung-Pan.
I'm an assistant professor at National Taiwan University.
I'm presenting Reality Rift's:
Wonderful Interfaces by Disturbing Perceptual Causality.
Here is an example of a Reality Rift.
The user turns on the faucets and sees the
wheel spinning as if there's water flowing out and
driving the wheel in different speeds and directions.
So the key here is that the system behaves
physically plausible explicitly without water.
This guides the user to naturally imagine
the missing water flow and thus elicits a sense of wonder.
Talking about wonder, there has been a long philosophical
discussion about wonder, which can be dated back to Aristotle.
He said it is through wonder that man now
begin and originally began to philosophize.
And in terms of interaction, before we
philosophize, we explore our physical reality by interacting with objects and
observing phenomena with different sensory cues like visual, auditory, and haptics.
We then form expectations in our imaginary
reality from the perception of causal relationships between observed phenomena.
And this is the stage where we feel a sense of wonder.
However, we no longer wonder once all
observable phenomena match our expectations.
And with media technology like language,
text, image, et cetera, we create virtual realities to convey our imaginations.
However, since anything may happen in the virtual world, we suspend our beliefs,
meaning that it does not change our fundamental beliefs in the real world.
And with computing machinery, we are able
to not only observe more in the physical reality, but also affect it.
And as computing machinery gains more
control over the physical reality, for example, programmable matters, new
fabrication machines, metamaterials, etc, we can create an interface to induce a
sense of wonder and make the user believe the new reality.
To induce a sense of wonder, we borrow the concept from Anders Schinkel, who says
wonder thus implies a shift in perspective that is particularly manifest when
something ordinary is suddenly seen as extraordinary.
So the key idea behind reality rifts is to
hide certain physical components in the imagined reality.
In other words, they only exist in the imagination and are invisible.
In the meantime, the remaining physical
outcome still happens in the surrounding environment,
making the user feel something ordinary is suddenly seemed as extraordinary.
And here's another example of a reality rift.
The user loads and launches the catapult.
And even though there is no projectile, the water in the tank still splashes.
And as if there's a projectile being thrown into the water.
And here's another example.
Note that the fan is not powered as you can see the blade is not spinning.
But the wind vane, it still turns just like there's a wind blowing it.
And here's another one.
And here we use flameless lighter, by the way.
And this example also showcases the simplicity of creating a reality rift.
Alright.
So hiding more components yields lower
level of affordances, but more general to be used in multiple scenarios.
So you can see here the general stick can
be used as a screwdriver to screw the screw.
You can see the motion of the stick
actually corresponding to this motion of the screw.
And here we can use that as an axe to break the wooden plate.
So we have shown six characteristic prototypes of reality rifts.
Now let's talk about how to open rifts in reality.
So to open the rift in reality, we took it
as a physical interaction and convert it into a causal graph.
So all interactions start with the user
input and end with observable effects to match the user's expectations.
In between, there's a sequence of intermediate phenomena that the
interaction can be further broken down into.
And by selectively hiding one or more intermediate phenomena, we can test to
which extent the user still can recover the original causal graph.
In the meantime, we have to find an
alternative cause to achieve the effect without being noticed
so as to maintain the original causal impression.
Let me use our catapult as an example.
So the user input here is to spring load
the catapult and the intermediate phenomenon we want to
put into the Imagined Reality is the appearance of our projectile.
And the final effect is the water splash.
And we use an air compressor to burst out
an air bubble from the bottom of the water tank to simulate a splash.
And we described the complete three steps of opening up reality rift in our paper.
Here, I'll just quickly run through it.
So the step one is to pick a causally perceptable interaction.
And we use spatial and temporal contiguity
to pick up a causally perceptablle interaction.
And in short, more mundane interaction is more causally perceivable.
And step two is to disrupt perception causality by hiding phenomena.
In our prototypes, we hide the object's
appearance as it directly infers the existence of the object.
And step three is to restore outcomes by alternative systems.
And here's an alternative system you just saw for simulating water splash.
And more details you can find in the paper.
So we wrap all six reality rifts
prototypes and their baseline counterparts as a one week preliminary field deployment
in a local escape room studio with 50 participants.
And here's a short clip of our deployment.
Alright. So we ask participants to pick their most
wonderful and wonderless interaction in reality rifts.
And these results support our hypothesis of the wonder clarity spectrum.
And you can find the spectrum in our paper as well.
And we also compare it with the baseline condition where everything was revealed.
And reality rifts did generate a more
enjoyable experience and trigger more user curiosity than the baseline condition.
Alright, so I would like to talk more
about the insights we found in doing this project.
The more realistic simulation and perfect camouflage, the more technical limitations
to realize reality rift, similar to all the mixed reality technologies.
For example, a better implementation of our wind vane would be to only hide the
wind in imaginary reality with the fan powered on and spinning as normal.
And this will be difficult to achieve because the blade of the fan needs to be
balanced in order to achieve these aerodynamics.
And while there are physical constraints, we see that the reality rift inspires new
technical challenges to connect sensing and actuation systems.
And reality rift decoupled those original
causes from their effects and opening up a design space for systems
that simulate effects while hiding alternative causes.
And with more and more new sensing and actuation technologies being developed in
recent years, we see reality rift as a source of inspiration for reimagining new
physical displays that gain control over physical matters.
And finally, I would like to bring you back to the beginning of the talk.
We have explored reality rifts
using computing machinery to create a physical interface to induce the sense of
wonder and bring the user into the new reality.
And our three step process eventually
comes down to understanding deeply the user's perceptual causality, and the
noticeable of the new imaginary reality, and the technical challenge in simulating
effects and camouflage the alternative sensing and accuracy system.
So I hope this paper makes you wonder
about the next new reality and the new tech within your own perceptual causality.
Alright, I would like to thank my lovely
students, Yi Chen, Yi-hao, Elise, and Tinyi.
I would also like to thank EnterSpace for
lending us space for the field deployment and providing feedback.
And also thanks Christian from ETH for chiming in for the last couple of months.
And thanks everyone for listening.