Hi, my name is Ryan Ingebritsen and I’m here to talk today about
Kinesthetic Empathy in Remote and Interactive Performance.
First of all we have to define what kinesthetic empathy is.
In kinesthetic interaction design it means the ability to detect and decode
The input of other users
Vicarious performance and inner mimicry in music and dance
talks about the ability to feel a performance while you are watching it in your body
Playing and breathing together in live performance can also be seen as a form of kinesthetic empathy
Research done by Alexander Demos et all in entrainment in musical performance
Suggests that when two musicians play the same music together
There is a correlation between their postural sway that is far greater than chance
and the same thing is true for performer and observer
when an observer feels a performance is expressive there’s a higher correlation
between the postural sway of the performer and the observer.
this seems to give experimental evidence for the existence of kinesthetic empathy in live performance
On remote platforms this ability to feel one another is somewhat blocked.
The slight time lag on remote platforms is generally blamed for the inability to play music online together
Even in situations where time delays can be eradicated
other missing elements also contribute to the ability to feel the experience of making music
such as peripheral vision and spatial localization of sound
And even very expensive experiments to try and emulate these conditions
have not reproduced a fully live feeling
and further these are inaccessible to the typical musician
networked performance that facilitate modes of performance native to the system have been with us for some time
it has been show how in such network performance, performers can achieve
kinesthetic empathy according to the definitions in kinesthetic interaction design.
we feel that because this kind of kinesthetic empathy is facilitated through technological mediation
and not necessarily physical presence, this could translate to the remote environment
So a few examples, first of all, Sentient Object
is a remote performance system for up to eight players
where data related to pitch, panning, tempo and duration of notes in a randomized tone generator
is sent over the web, then sound is generated locally so that only small packets of
control data are sent and received, minimizing latency.
The design of the control system makes it fairly intuitive and the indirect nature of the
interaction renders any kind of latency unnoticeable.
Performers are encouraged to try to differentiate themselves form other players at first
until the understand the relationship between their own sound and those of others and then trying to explore other expressive combinations
Most players felt that they were interacting with other players in a way that felt responsive
and one performer said it was like playing music with others for the first time since the pandemic started.
another example, Layered Sound, is a live painting performance created by myself and painter Melanie Brown.
and its sound is used to generate a number of sonic responses.
Remote performance was achieved by using the ListenTo plugin to receive and send high quality sound between performers
The online environment was almost an advantage as it created an immersive sonic environment
emphasizing the connection between the brush strokes and the sonic reactions.
And finally in the summer of 2020 the Fresh Inc festival in Chicago
encouraged composers to create works specifically for an online environment
that did not require precise time synchronization.
The most successful cases involved some explicit compositional instruction to focus on a specific aspect of listening to facilitate connection between performers.
An example of this is the piece colorful clouds by spencer arias.
Performers were instructed to pay attention to the electronic score, the shifting color, and the density of the clouds
and react to each other based on deep listening to one another’s playing.
When thinking about how kinesthetic empathy was actually achieved, there were certain prerequisites we had not considered
When we learn to play an instrument, we try to train the body to react automatically.
Just like driving a car, this knowledge resides in what is often called the “body schema”.
Though playing different instruments, various members of an ensemble are familiar with
as a kind of interactive system. Through this system they can achieve kinesthetic empathy.
Though we have the term kinesthetic empathy to describe the connection between two human beings,
we have no common term to describe the consistent mapping between our auditory and sensorimotor system that allows us to play instruments.
We propose the term “auditory kinesthesia” to describe this phenomenon and suggest that it is a prerequisite for kinesthetic empathy.
In our three examples, Sentient Object achieved auditory kinesthesia
through a clear and easy to use graphic user interface,
Layered sound takes advantage of an existing auditory kinesthesia and
leverages that to create this kinesthetic empathy remotely,
and Colorful Clouds leverages the musicians existing auditory kinesthesia with their own instruments
but replaces the interactive system of traditional notated music
All of these examples were done in the context of live performance in real-world situations.
So for us the next step of this research is to put this in a more controlled environment
and for this we propose using another interactive system, the body sample player.
The body sample player is a system designed for previous works and has had a long performance history
with several pieces performed multiple times by skilled performers.
We propose using the experiments of Alexander Demos as a model for our investigations
to explore the correlations in perceived expressivity and correlations of movement between performers and observers
to see if there is a difference in perceived expressivity between experienced users and users with no first hand experience of the system
and if certain train strategies develop a greater sense of aural kinesthesia and result in more expressive performance.
By this we hope to further reinforce the observations in these examples
And that further studies in the fields of cognitive and neuropsychology will continue to inform our work as well.
Thanks for listening and now lets open it up to questions.