Translator: Mizuki Deguchi Reviewer: Riaki Poništ
You might be looking at this, and thinking about what you see.
But it’s actually possible to look at this and instead think about what you hear.
Think about music.
Some of its building blocks are:
"rhythm," which is just when notes happen over time with silent spaces in between;
it has "harmony,"
which is when two or more notes at different pitches
are played together at the same time
to produce a certain psychological effect or mood;
it has "form,"
which is how the chunks and sections of music are arranged over time.
And all those building blocks together create a composition.
Now, think back to painting.
This and visual arts also have rhythm.
But that's visual rhythm,
which can be created with marks, like this.
Boom.
And here, we have spaces in between paint marks
instead of silence in between musical notes.
It also has harmony, but that's color harmony,
which is the placement of two or more colors together
to produce a certain psychological effect or mood,
like this.
And here I changed the mood
with this contrast and harmony between blue and orange.
This image also has forms and shapes
that occupy chunks and sections of the image,
and all these elements together create a composition.
Also, when musicians add different chords or timbres or textures to music,
we say we are adding "color" to the music.
And it creates another tone or mood
just like if I added another color to this painting.
Boom.
So for example,
those could be my "conga drums," adding "color" to this "music."
Another common music term is "staccato,"
which is a series of very short notes,
usually played very quickly and with just short silences in between.
And we can actually see
how this same effect can be created in painting,
like this.
And you can actually see staccato,
and you can hear the sounds that marks being made as I make them,
like this.
Ch-ch-ch-ch-ch-ch-ch-ch-ch.
We can also say this painting is syncopated.
"Syncopation" is the emphasis of silences between musical notes,
but again, here, we have spaces in between paint marks.
So we can actually "see" staccato and "see" syncopation.
But it's crazy
because this is something that's only supposed to exist in the world of sound.
But we can actually see it; we can "see" into the world of sound arts.
Something else compelling is that legendary musician Miles Davis
used to look at Kandinsky paintings like these.
And he would compare their balance and juxtaposition of lines and shapes
to the balance and juxtaposition of chords and sounds in music.
And he said, "If artists like Kandinsky can do stuff like that with images,
I can do the same thing with sounds."
And he said,
"A painting is music that you can see,
and music is a painting that you can hear."
And the key with that
is whether it's with music or painting or any other kind of art form,
all those building blocks are having an impression on us,
and they evoke deeper understanding and insight
into whoever or whatever created them.
And regardless of which building blocks are being used or not in any given work,
rhythm is the one that's always there and still communicates on its own.
For example,
in this black and white painting,
even without color harmony, or very much form or composition,
we can still see that the "rhythm" of the marks is aggressive,
and we know that the artist carried out aggressive gestures to create this image.
And we can sense an intense energy coming from within the artist
and out through this work just through visual rhythm.
And you can look at this painting and say that it's probably not a happy one,
just by sensing its rhythm.
Now, think about filmmaking,
which is a very powerful combination of music and sounds and visuals,
but with that added element of motion -
and of course motion itself is rhythmic -
and drama is created through the rhythm and timing in film.
Think about when you're watching a movie,
and you see two characters having a conversation.
First, you see one person on the screen talking.
Then it switches, and you see another person on the screen,
then you see the other person, and it keeps going back and forth.
The precise timing of when to make the switch
is called a "beat."
And again in music, a beat is used for rhythm, and usually percussion.
But in this case, it's used to signal and signify a key moment in the story,
for example, when a character realizes something,
or the exact moment that the character has a reaction,
or when there is a pause.
So here in film,
rhythm is used to communicate something subtle and intuitive.
And even the writing of the story line has rhythm,
because really a story is just things happening over time,
and if you think back to music,
the rhythm is just when the notes happen over time.
And both have their own culminations and buildups
and different tension and release.
And also when writing,
you have actual rhythmic structures such as "iambic pentameter,"
which you can find in Shakespeare plays and sonnets,
and that's called "meter," another music term.
And you also have the writer's "voice,"
which is the way that any particular writer
puts together phrases and words and syllables
almost as if there were shapes and lines to create a certain psychological effect.
And a finished piece of writing is also called a "composition."
And that’s all in writing, but it’s just like in music.
And we can continue to detect how rhythm is used across other artistic disciplines.
For example, architecture,
and various design,
and of course dance, which is all motion,
where rhythm is used
to very directly communicate and express so much.
Through rhythmic gestures,
we identify with the creator of those rhythms in a deeper way
that cannot just be explained in words,
and that’s part of the purpose of the art
because if you can just explain it to somebody,
then why make the art, right?
Attitudes, feelings, emotions, thoughts and energy combine into something unique
in the artist’s mind that gets transferred through the work.
But, everything has rhythm, not just arts:
it’s the way that we move, walk, talk, breathe, think, blink, everything.
Every little thing that happens has rhythm.
Miles Davis also mentioned the "music"
in the rhythmic culmination of punches in boxing,
and the timing of punchlines in comedy,
and in the buildup of footsteps leading up to shooting a basketball
that has tension and release.
He said, "Rhythm is all around us.
Even if you stumble, you might want to play that."
So these connections in insights through rhythm within the arts
can actually be applied to everything else.
And as an artist,
I wondered what would happen if I did take everything around me
and turn it into, as Miles would say, "music that you can see."
So I created drawings and paintings
based on the sounds and movements that specific people around me made,
and in other words,
the compositions that they created just by being alive.
This one is based on time
that I spent with Miles Davis’ last drummer, Ricky Wellman,
who was a friend and became the owner of the only prints of this,
and it was one of the very first in this exploration.
This is based on my uncle, who taught me how to play music.
I took this a step further,
and while thinking about the rhythm of everything around me,
I started to think about the rhythm of how I perceived everything.
And I started to focus on how the rhythm of perception
could tell me something about the nature of perception itself.
So, I created a video projection for the venue at the TED2016 Conference,
which was themed "Dream."
And in the context, I could explore and work
with perception and consciousness in this way.
And the video was about the fragmented stages of dreaming
and the rhythm of how sounds and visuals interacted in a dream.
So, I could go from doing paintings like this
to making videos like this,
and activating them with more motion and explicit rhythm.
(Electronic sounds)
(Diegetic sounds)
(Electronic sounds)
(Diegetic sounds)
(Electronic sounds)
(Diegetic sounds)
So, I started to get closer and closer
to how rhythm can open insight into anything.
And, we can see
how this spreads into other areas of how we understand the world.
For example, music is mathematical,
and math can also be translated
into very beautiful patterns that are also very rhythmic.
Also in physics, at CERN, one of the world-leading physics labs,
they have a residency
where arts and physics are meant to inspire each other.
And at the lab, they have some of the biggest machines ever built,
where they smash particles together to discover new particles and new physics.
And there was a project called "disCERN,"
where real-time data from those collisions was turned directly into music.
And I was so fascinated by what was going on at CERN
that I decided to visit, and it was my first stop in Switzerland,
and this is what I saw.
This is a cloud chamber,
which reveals the trails of actual particles
as they come from space
through the atmosphere and through this chamber.
And they pass through our bodies, they pass through objects,
and straight through the earth almost totally unaffected.
And guess what I noticed first:
their rhythm.
And I wanted to identify with them even more
in that intuitive and artistic way,
and get the same kind of sense for it.
So I created the virtual reality simulation of those particles,
not at the exact same speeds
because some of them are impossible to replicate,
but at least proportional enough
so that it had a similar rhythmic and visual composition.
And it had the same rhythmic impression on us when we put on the headset.
In this documentation, what you see on the back wall
is what the person wearing the headset
actually sees, experiences and moves through.
So it's an immersive engagement
with the rhythmic system and identity of these particles.
So next time you see artwork, think about what you hear.
And next time you hear music, think about what you see.
And by tapping into how every little thing within and around us
is like artwork and music,
tap into its rhythm
to identify with it and connect with it in a deeper way than you ever had before.
Thank you.
(Applause)