Hi my name is Courtney and today I'm going to present our paper
"Negotiating Experience and Communicating Information Through Abstract Metaphor".
In HCI we might typically think of metaphor, for instance, as the desktop metaphor wherein
computer processes are represented by an icon that helps to describe its functionality with
something from the real world. For instance the bin on the computer desktop is used for the action
of deleting files from the computer's memory, or through WIMP, Windows Icons Menus and Pointers
wherein selection tasks and grouping of information is done as it is in the
physical world through a collections of items on a desk, and openable and pointable menus.
These representations are really helpful in communicating information and
aspects of processes and tasks in the digital world between the human and the non-human
agents within the computer system. However these one-to-one mappings of taking a digital process
and representing it with a physical action from the real world do have limitations.
For instance, meaning can actually be achieved without a real world reference and this is
sometimes often necessary. A funny example of this for instance is that younger computer users
today still use the save icon as a floppy disk, although many of them have never seen a floppy
disk in real life, so do not actually have the association of the floppy disk as being a saving
action or a file storage action but rather have still been able to associate the meaning
of saving with this icon. As well, there are a lot of abstract experiences that we have that actually
have no real world reference in general. Such abstract experiences through different sensory
communication are very hard to talk about, so we don't actually have a good way to represent
these even within language. We might be limited by vocabulary or not have any physical representation
of these experiences and so they're often something that we understand internally or
just through knowledge of our body and in that sense we all have a different experience and
different backgrounds so our understanding of these very abstract experiences are going to be
very very different from person to person making them more difficult to talk about.
However humans are quite good at conveying experiences and they do this through abstract
metaphor. In this study we explored this kind of abstract metaphor in the vocal lesson. The voice
lesson is a really really interesting place to explore this because it relies a lot on individual
internally sensory experiences of the body while singing, and the goal of a voice teacher in their
lesson is to take their understanding of their body and their practice and get their students
to actually produce their own actions with the desired result in terms of their sound
or their behavior. This model provided by Dunbar-Wells in a pedagogy context
demonstrates that the teachers' metaphor is translated into sensory-based imagery which
then eventually results in physical adjustments and changes to the students tone outcome or other
aspects of their vocal behavior. So given that humans are quite good at communicating sensory
experiences, for instance in the voice lesson, we have a few research questions when it comes
to metaphor and HCI. Number one, we explore how humans are able to use metaphor to reach
mutual understanding and we do this by exploring voice lesson. Number two, we want to know how
metaphor actually works to communicate subjective knowledge. Number three, we want to know how these
elements of metaphor can be operationalized within HCI and communication especially with non-humans.
Number four, we want to know how metaphor can help to address individual knowledge.
So the main contributions of our paper are that we will provide a model of metaphor translation
between two agents, whether human or non-human. They expand on the concept of metaphor within
HCI to include many different representations of information in different modalities. And then we
show an application of our model to explain communicating in existing design and future
paths that use this kind of information representation. Within our methodology,
we worked with 12 voice teachers who were teaching in a variety of styles. The first part of a study
engaged with their imagery abilities through the Bucknell Auditory Imagery Scale and the
Movement Imagery Questionnaire with the goal of seeing does a teacher's imagery ability impact
why they choose certain metaphors. For instance would a teacher with better ability to produce a
visual image be expressing themselves more often through visual imagery or through visual metaphor.
Then secondly we use semi-structured interviews to talk about vocal fundamentals, how the teachers
teach them and how they learn these fundamentals themselves. The fundamentals included supported
breathing, body posturing, controlling tension in the body, and vocal resonance. The first thing we
examined were those imagery abilities and whether or not they correlated with the metaphors that
the teachers use. Using a Spearman's ranked correlation tests, we found that there was no
correlation between their metaphor abilities and the metaphors they used. So the why these
metaphors are chosen must be due to some other reason. To uncover this, we used a thematic
analysis to look at the metaphors they used and their teaching strategies. We examine how these
metaphors work to convey knowledge. From this, we uncovered four key themes as to how the metaphors
were functioning within the voice lesson and more broadly. We found that metaphor works
because it relies on embodied experience rather than pre-existing or domain-specific knowledge.
So we take knowledge that does not have any relevance to the voice lesson for instance,
comes from other life experience and we then apply it to a very domain-specific knowledge.
Metaphor works independently of language and concepts can be expressed in many forms. So
because the concepts are rooted in a higher level cognitive understanding, the actual language
used to represent that understanding doesn't matter. As said by many of the teachers who were
speaking English as a second language and in fact, learned vocal pedagogy in their native language.
Number three, metaphor communicates only core components and limits unnecessary detail and
this is something that's chosen by the teachers communicating agent to help students rely on
their bodily understanding rather than details that might be sort of confounding or confusing.
Finally that metaphor uses intentional ambiguity by using references that can be interpreted
individually to incorporate individual lived experience and knowledge and allow people to
take their own approach to whatever that given instruction. There are a lot of details on
these metaphors and specific, and often very funny metaphors, that teachers use with their students,
so please do view our paper to see some of that and maybe get a free voice lesson.
In the end, with these themes we come up with a model of metaphor communication as shown here.
On the left side, we have a communicating agent in this case a teacher. It could also
be the the computer system or the interaction device. On the right side we have a receiving
agent in this case a student - it could also be an end user of a computer system.
So the two different colored bubbles represent their individual lived
experiences and understanding and in between we reach an idea of mutual understanding.
Metaphor negotiates information between the communicating and receiving agents using an
abstract reference wherein the communicating agent encodes their understanding into a metaphor based
on whatever their lived experience is. It is then given to and decoded by the receiving agent
or the student, who internalizes it into their own cognitive understanding relative to their
own lived experience. This process as you see is happening in a cycle or a loop and is flexible
and iterative. It's manipulated by both parties which makes it unique from how some other HCI or
more traditional HCI methods of metaphor occur wherein the human agent, or the receiving agent
in this case, is the only party responsible for understanding. Our model of metaphor communication
can explain designs both inside and outside of academia. For instance on the left hand side, in
A B and C, we see BumpTop which exactly mimics a real world desktop environment. Our understanding
of how to access documents doesn't come from the fact that they exist in real world or even visible
forms, but rather that we understand that by clicking on objects we can access them from
a menu. In fact this kind of representation might limit all the potential of the desktop environment
wherein we can store thousands of digital documents compared to our real world desktops
where we can only store it so much. On the other hand, on the right hand side we see an example of
how a metaphor has changed over time and humans are still able to understand and interact with
it. When the Android phone was first implemented, it included a physical button to access the menu.
This physical button then changed to a digital button, and then to an icon representation for
the menu before disappearing completely and being replaced with a swiping gesture. So the metaphor
for accessing the menu has changed over time. The metaphor itself can be negotiated between
designers as the phone's design changes and the end users who do still understand that there is
a menu that can be accessed from the bottom of the screen. Our model of metaphor communication,
and indeed the expansion of metaphor beyond just icons and symbols, can explain how projects such
as the Heart Sounds Bench are able to communicate information about people using auditory metaphors.
This is a good example of ambiguity and metaphor communication wherein because we
all have a different association with heartbeats in our life from sociocultural applications,
we're able to understand aspects about other people just by listening to their heartbeat
sound without providing any sort of objective data about the heart itself. We've created a
model of abstract metaphor communication which can be applied to a vast variety of metaphors and
explain why they might work in human-to-human, and also human and non-human communication.
We work with human agents and look at how their knowledge of an instance of subjective body-based
communication within the voice lesson can be applied. To working with non-human agents we
reflect on communication roles and the expansion of metaphors within the concept of HCI. And
finally we provide the beginning of a framework to evaluate metaphor and communication in design.
Thanks for your attention and we look forward to getting in touch with you
about metaphor communication and other projects wherein metaphor might be useful.