Hi and thanks for tuning in. This is Feeling Colours: Investigating Crossmodal Correspondences
Between 3D Shapes Colors and Emotions. My name is Oussama Metatla and my co-authors are Anan
Lin, Meike Scheller, Feng Feng and Michael Proulx we're from the Department Computer Science at
the University of Bristol and the Department of Psychology at the University of Bath in the UK.
So in this paper we show evidence for crossmodal correspondences
between tangible objects visual color and emotions we asked people to feel 3d printed
shapes and to assign a color and an emotion to their tangible experience
the work is grounded in crossmodal correspondences these are non-arbitrary perceptual associations
between sensory stimuli. They've been extensively studied in areas such as cognitive science and
experimental psychology and numerous examples of crossmodal interactions have been documented.
We know for example that people often tend to associate larger objects with lower pitched
sounds and smaller objects with higher pitch sounds. This particular crossmodal correspondence
could have arised from natural correlations of physical properties. For example that larger
objects resonate at lower frequencies and vice versa. But there are other types of crossmodal
correspondences that are more abstract and they cannot easily be explained by physical properties.
You can try this out for for yourself right now. If I was to show you these two shapes on the screen
one angular and jagged shaped like a piece of shattered glass another one is round uh like a
cloud and then I gave you two random labels "Bouba" and "Kiki" and asked you to assign one label to each
shape, which one would you assign to which? So the majority of people would assign the label "Kiki"
to the jagged shape and the label "Bouba" to the round shape. This is a famous visio-linguistic
crossmodal correspondence that actually have been shown to hold across a range of
age groups cultures and so on. So why is this relevant? In short because in HCI we're
interested in multisensory interaction and it's not only increasingly feasible to
incorporate more and more sensory modalities at the user interface and user experiences
but it can also be more beneficial to do so. For example we can improve immersion
or engagement or accessibility. What crossmodal correspondences tell us is that we ought to be
careful and mindful of crossmodal effects because the way we perceive and interpret information
from one sense can be influenced by information coming from another sense.
So in this paper we wanted to look at the question of whether there are crossmodal correspondences
between 3d shapes, color and emotions independently of vision. Whether the way we feel tangible objects
combine with color and influence our emotional judgments. So we started off from the Bouba/Kiki
paradigm and 3d printed bouba-like and kiki-like shapes and we varied them in terms of levels of
angularity and complexity, so we operationalize this in terms of the number of protruding points on
the surface of the 3d printed object. We recruited 30 participants we hid the objects away in a box
so they couldn't see them and ask them to feel the objects and then to choose a color from a palette
of colors and also adjust brightness and also to assign an emotion in terms of an answer to the
question feeling these objects give me sense of and then choosing a term that captures
the pleasure arousal and dominance emotion model. What did we find? In terms of colors we found
that rounder objects and less complex objects tend to be associated with higher brightness
and the color blue and angular objects and more complex objects tend to be associated with lower
brightness in the color red in terms of emotion we found that rounder and less complex objects
tend to be associated with higher levels of pleasure lower levels of arousal and more of a
sense of being in control whereas angular and more complex objects were most often associated with
lower levels of pleasure higher levels of arousal and more of a sense of being dominated or having
a lack of control. Importantly these findings contrast those reported from similar studies
done with 2d shapes or 2d images. So this means that designers cannot simply extrapolate potential
perceptual and interpretive experiences that are elicited but 2d images to seemingly similar 3d
tangible objects. So other principles are at play and direct extrapolation can lead to undesirable
outcomes. We extracted four association strategies using thematic analysis one that often refers to
geometric features explicitly another that refers to familiar objects and familiar experiences
a third that relates to the personal sense of aesthetics or pleasantness and a final one
that looks at the coherence of choice within the boundaries of the experimental procedure itself.
So based on these findings we articulated a design space in the paper
to help create tangible multisensory artifacts that can trigger specific emotional precepts
and with this we think can help provide concrete actionable
recommendations on how to exploit crossmodal correspondences in HCI. That's it. Thank you.