Welcome everyone to this pictorial presentation entitled: Designing Physical Artifacts for
Tangible Narratives: Lesson Learned from Letters to José. My name is Daniel Echeverri and i'm also
presenting on behalf of Huaxin Wei. We're both from the School of Design at the Hong
Kong Polytechnic University. The purpose of this pictorial really is to reflect upon the
design and making of Letters to José, which is a tangible narrative that we built during
the last three years as a practice led research project. The purpose of our paper really is to
integrate different concepts that we brought from the design of tangible user interfaces
but also aspects brought from theories of narrative and interactive narrative.
This presentation is going to be organized in the following way: first, we're going to start with
a quick overview of tangible narratives and then we're going to present Letters to José.
Second we'll discuss some of the creative explorations that we did during this research,
and finally we conclude by proposing a visual typology that can describe and
define artifacts for this type of narratives. If we're going to characterize narratives,
we could say that they are a form of interactive digital narratives. Usually in the world of
Human-computer Interaction, they are described as interfaces, as platforms, or as environments,
but ultimately, the purpose of this type of narratives is first to construct a story
world where there are interfaces, objects, where there is an environment or system environment,
and where there's narrative multimodal content. The purpose of technology in this type of
narrative is to support the narrative itself and allow, somehow, through interactivity,
allow people to create the story and to play with different aspects of the narrative. If
we're going to talk specifically about Letters to José, this picture shows some of the panels,
some of the paper worlds, as we like to call them. Letters to José is a non-linear narrative,
inspired by a set of letters, a group of letters, that Jesús, who used to be my grandfather and his
brother José, wrote to each other during the late 1940s. As a hybrid experience, in the sense that
it makes use of both physical and digital means, this tangible narrative is presented as three
physically unfolding story worlds or paper worlds like the ones that we showed in the previous
slide. They combine different paper mechanisms with visual, performative, and auditory features.
If you want to see more information about Letters to José you can scan this QR code and it will take
you to a video with more details about the experience. Now, if we want to talk about
creative exploration, for the past three years we designed Letters to José, and we could argue
that there were three type of explorations. One is material exploration; we tried to find ways
to make things and so this is where we talk about using different materials, different techniques,
different substrates; how do we achieve, how do we build specific objects, ultimately leading to
early prototypes of the experience. In the second type of exploration, referred to it as functional
exploration. It was more about how do we use the artifacts, what are the functional considerations
in the way the relationships between the movable artifacts and some of the panels, as the one that
you saw in the previous slides, they can be a representation of the story world. And finally,
interactivity exploration, was more of how do we convey the story while people
engage with the narrative. In certain way, through interactivity exploration,
we figured out ways to make things mean. Through interactivity, the user, the interactor was able,
not only to convey actions, but also to perform meaningfully with the narrative.
Through these methods such as prototyping and annotating and sketching, we created systematic
evidence of our research process and our design process. Through that experimentation we created
new knowledge. Obviously that knowledge came along with the empirical study that we conducted,
with existing literature. So, from the perspective of tangible user interfaces
we were able to explain how the artifact bounds to digital content. From the narrative,
if we look from perspective of narratology, the purpose was more to understand what the role
of the artifact in the narrative was. But there was a theoretical void that we noticed. And is:
how can we describe these artifacts, and how can we describe their qualities.
So based on our evidence including our study results, we noticed that we could describe these
artifacts in a tangible narrative system, in a more generalized and systematic way.
While existing theory on tangible interfaces could describe some of these properties of
the artifacts, some of the models, and some of the existing frameworks, fall short, especially
when seen from perspective of narratology. Based on our reflections, based on the empirical study
results, and obviously, existing narrative theory, we frame the artifacts that have a
narrative purpose beyond system operative functions, as artifacts for storytelling.
These artifacts can be described in three qualities: first the artifacts' diegesis,
second the artifacts' embodiment, and third, the artifacts' function in the narrative.
Each quality can be described using different degree or indicative levels
or values. To put in a better way, there is a real world inhabited by an interactor.
Inside that real world, there is a presentation space. That presentation space is
where everything the experience offers and where everything of the experience is happening.
Inside the presentation space, there's a story world that is inhabited by characters,
that describe different events, and that people will engage with.
Inside the presentation space, the real world, the story world... there are different artifacts which
people can engage with. Some of those artifacts are placed inside the story world, some of those
artifacts are placed in the presentation, some of these artifacts are placed in the real world.
Whichever way, we describe these artifacts on three qualities.
The first one is the diegesis of the artifacts which describes in what way the artifact is
placed in the time and space of the narrative. So we have diegetic artifacts; they exist in the real
world and they exist in the in the story world in the same way and represent the same thing.
There are trans-diegetic artifacts; which somehow bridge the story world with the presentation or
the real world, and they are represented differently. And extra-diegetic artifacts,
which are outside of the story world. They reside only on the presentation space.
Then we have the function of the artifact. If it's ontological, it means that the artifact
has a real impact on the story world and can change the plot, for instance. Exploratory
artifacts allow the interactor just simply to explore different paths, but it does not have
an impact on the plot of the story. Then we describe the embodiment of the artifact, meaning,
how close is the action and the outcome of that action. So we have fully embodied artifacts,
where the action and the outcome is manifested in the artifact itself.
There are coupled artifacts in which one artifact is coupled to another and the outcome happens
right there, where that coupling is happening, and then non-graspable embodiment, which refers to a
relationship that is outside of that space where the action happened. It's completely disconnected.
Now, the relationship between artifacts and the environment are crucial in the way people perform
and people understand the narrative and the story. While this relationships are
conditioned by artifacts and the qualities that we described in the previous slides, the
embodied and tangible interaction method used or applied to an artifact also has a critical role.
So, how do we describe those methods? We could say they are space directed,
in which the artifact and the user or the interactor, negotiates a space.
Body-directed, in which the artifact creates a relationship with the interactor's body,
so it can be wore, or can be you know, moved through with the hand, for instance.
Or artifact-directed, in which the artifact has relationships to itself, as things that can change
in the artifact, or that has a relationship with other artifacts. These methods, they
mediate between the interactor and the artifact itself. Ultimately, what happens across these
artifact for storytelling, and the way people engage with them, fosters empathy, it foster
identification of the interactor with some of the events or some of the characters described,
it fosters the immersion into the narrative, it helps the development of, you know, mental imagery
and people imagining what happens in the story, and ultimately leading to a sense of reward.
Through this typology, we look to unify concepts brought from tangible interaction and narrative,
and we hope that these two typologies can guide the future work in the design of tangible
narratives. We intend, or we look to provide authors in designing with just very basic concepts
that can help them design engaging narratives that go beyond the two-dimensional, the bi-dimensional
world, into what the real world is. Tangible storytelling stands out from other forms of
digital narrative, because it is about playing with real tangible things, real tangible objects
that exist in the real world. These tangible narratives mean that the designer and the
author needs to find ways for people to play make believe with those artifacts and with the
narrative. Artifacts for storytelling are simply props, but they have a very particular purpose,
they have very particular imaginings: they make the fictional world, they make it real.
This concludes our presentation. Thank you very much, feel free to reach out to us via email.
Thank you again, and we look forward to hearing from you.