- This is a talking piano,
and it's so cool.
- Hello!
I am a talking piano,
and I'm so cool.
- And his name is Chopstix.
(upbeat piano music)
So in addition to talking,
he can play simple songs like that,
but also like, really complicated songs.
(dramatic piano music)
So today, not only will we talk
about the science behind making a piano speak human,
but later, he's gonna play the drums for us and sing for us.
Not yet, Chopstix.
Remember, for my videos,
that song usually comes a little bit later.
- You just assume I watch your videos?
(upbeat piano music)
- Anyways.
Then he is gonna attempt
to play the world's most complicated digital piano song
from Sheet Music Boss, "Rush E."
(chaotic piano music)
There's a reason this song has never been played
with an actual real-life piano,
so I've legitimately got a couple fire extinguishers
on hand in case the whole thing goes up in flames.
- Did you say flames?!
I swear, if you so much as make me warm,
my lawyers are going to sue your-
- But before we get to any of that,
we need to first understand the really cool engineering
behind how pianos even work.
'Cause if you look at the guts
of this modified see-through piano,
for something that was invented
by an Italian dude 300 years ago,
you can see it's pretty complicated.
In fact, if you zoom in and just isolate on a single key,
this is what's happening every time you push down
on the white part.
And if you're thinking like an engineer,
you know the best design is always the one
that gets the job done in the most simple way possible.
And when I first saw this,
no offense to Bartolomeo Cristofori,
the inventor of the piano,
it just seemed overly complicated.
I mean, the first piano I ever played
(bright piano music)
was pretty dang simple.
Like, why can't you just have a class one lever
where when I push down here,
there's a fulcrum in the middle,
and it hits the string here?
Well, it turns out there's two problems
with that right off the bat.
The first is that the hammer remains
in contact with the string When you hold the key down.
That means the string isn't free to wiggle and vibrate,
which is how the sound is created.
Like, with this guitar,
(string vibrating)
it sounds great until you touch the strings.
And now because it can't vibrate, you don't hear anything.
In fact, you can see
(metal clacks)
that's exactly the problem with this toy piano.
The second problem is this is an upright piano,
so the strings must be vertical to fit.
But that's an easy enough fix
if we shift them here and add an extra hinge point.
However, you'll notice we're still damping the string.
And as you can see,
(string vibrates)
the piano should continue making sound,
even if I'm holding the key down.
So how do we do that?
Well, Bartolomeo got real clever
and added a little extra hardware here.
So now when you press the key,
this jack flings the hammer forward
so it hits the string,
but then the jack gets its foot caught,
which moves it out of the way
so the hammer can move back and not dampen the string.
Fun fact:
Most people think pianos are considered string instruments,
but this hammer-like action
is why they're actually classified as percussion.
(bright piano music)
Okay.
So now if I press the key down,
it will continue to make noise.
But wait, if I let my finger go,
the sound should stop.
So something is damping that string.
But if you look in our current build,
you could see if I let the key go,
there's nothing to stop the string
from just continuing to vibrate and make sound.
To solve this, Bartolomeo added an extra pivot point
and some more hardware.
So now you see this damper rests on the string
and keeps it quiet until you press the key down.
Then the hammer strikes.
Then it's free to vibrate
and create noise all the way up
to the moment the key is released.
And finally, if I wanna play a note really quickly
and repeatedly like this,
(simple piano music)
I need the hammer to be close to the string
and not way back here like this.
So by adding this backstop,
it keeps the hammer close after each hit
in case I need to quickly hit it again.
So there you have it.
Apologies to my fellow engineer, Bartolomeo.
It turns out it's only as complicated as it needs to be
and no more.
It's also sort of simplistically beautiful
that all that functionality is from only one input,
(string vibrates)
every time you press a single key.
Now, Chopstix here is a special case,
not only because he's see-through,
but because he can play himself.
And it is true pianos that play themselves have been around
for over 100 years,
but there's an important difference here.
An old player piano works
by scrolling through a paper sheet like this,
and when it encounters a hole,
it passes air through, which plays the key.
So all the keys are played at full force,
and the timing isn't very precise.
Whereas Chopstix here is a modern version
that uses a solenoid to actuate the key.
We place the solenoids right here,
so when the rod extends,
it's no different than someone actually pressing the key.
You can see them all lined up along here.
But with solenoids,
not only do you get your timing down
to fractions of a second,
but you have 127 different levels of increasing force
to press a piano key,
which means you can make perfect recreations
of a human player.
So here's my friend Andrew
from the YouTube channel Sheet Music Boss playing
a simple sea shanty, and then recording it as a MIDI,
and then Chopstix uses that MIDI
to play it back perfectly.
Only since he is not limited to a mere 10 human fingers,
the songs can get more complicated.
(upbeat piano music)
But he can do even better than that
because those 88 precision solenoids allow him
to make crude reproductions of entire bands.
See if he can pick out the singer's actual voice
in this classical piece of music history.
(upbeat music)
♪ Never gonna give you up ♪
♪ Never gonna let you down ♪
♪ Never gonna run around and ♪
♪ Desert you ♪
♪ Never gonna make you cry ♪
♪ Never gonna say goodbye ♪
♪ Never gonna tell a lie ♪
♪ And hurt you ♪
- And so now that we know
at least partially what he's capable of,
before we attempt "Rush E,"
let's quickly discuss how we actually make Chopstix speak
and sing.
Ahem, Chopstix.
- Oh, sorry.
(gentle thoughtful music)
- That's better.
And to do that, I'm gonna pass it over to my buddy Grant
from the YouTube channel 3Blue1Brown
for a simple explanation.
- [Grant] The signals from speech can be visualized
with a waveform, which you might think of
as telling you the air pressure next to a microphone
as a function of time.
If you zoom in on a little window of it,
you might notice it looks like a rhythmic repeating pattern.
As it happens, one of the most delightful facts
from math tells us that for pretty much any signal,
you can express it as a sum of pure sine waves,
which in the context of sounds correspond to pure pitches,
higher frequency sine waves giving you higher pitches.
If you want to perfectly reconstruct the original signal,
this often requires adding a very large number
of pure sine waves, each with a different height.
Now, a favorite tool in signal processing
is something called a Fourier transform,
which you can think of as telling you the heights
of each one of these sine waves
as a function of the frequency you're dealing with.
So for our piano project,
if you take this Fourier transform
and you just consider the peaks,
it's a way of giving you the most important notes
to be played to recreate the sound.
If you add just a handful
of these dominant frequencies,
you'll get a signal which is almost,
but not quite, the original signal.
And if we do this process
across many different little windows of our full signal,
it's a way of telling us what notes should be played
at what time and with what level of intensity.
And if you do it right,
you can get something that sounds like this.
- Hey Chopstix! repeat after me!
- Now, I do have to come clean
and admit that I am cheating a little bit
by putting the words Chopstix
is speaking up on the screen.
As humans, 80% of the information our brains gather
about our surroundings comes from our sense of sight.
And so as our dominant sense,
our brain really leans on our eyes,
even when processing what we hear.
If you don't believe me,
here's two tests you can do right now.
Number one, go back and re-watch any part of this video
where Chopstix is speaking.
Only this time, close your eyes,
and see how much harder it is to understand him.
And number two, maybe you've heard this audio clip
from some random '80s toy saying the word "green needle."
- [Toy] Green needle.
- Now compare that to this other clip
that says "brainstorm."
- [Toy] Brainstorm.
- Only as you might have guessed,
those are the exact same audio clips,
which is bonkers because those two words don't even have
the same number of syllables.
For proof, I'll repeat it twice more,
only this time you can randomly pick a word
to look at each time.
- [Toy] Green needle. Brain storm.
- All right. And so now for the big finale.
In a world's first exclusive,
I present an actual piano attempting
to play the world's hardest piano song, "Rush E."
- If anything should happen,
promise you'll take care of my bench.
- Of course, buddy. Good luck.
(chaotic piano music)
- [Announcer] E.
(chaotic piano music)
(chaotic piano music continues)
- That was incredible!
(dissonant piano music)
- The biggest problem with me making a YouTube video
to try and teach you something
is that it's just not very interactive,
which is why I'm pleased to say my friends
at brilliant.org,
who just so happen to be supporting this video,
have solved this issue.
And if you don't know,
Brilliant is a problem solving website and app
that has over 60 courses,
including topics in math, science, and computer science.
But their real special sauce
is all about making the content interactive,
which is way more effective at building your intuition.
My favorite one so far is their scientific thinking course,
where instead of memorizing specific formulas,
that interactivity builds intuition
about general principles you see
in the everyday world around you.
And it works great on both desktop and mobile.
So instead of picking up your phone
and mindlessly scrolling a newsfeed
that leaves you feeling sad,
with Brilliant, you get this addictive experience,
which leaves you feeling stoked
by unlocking the hidden principles
that govern our amazing world.
So if you want to grow your brain
and support my channel at the same time,
go to brilliant.org/markrober,
or use the link in the video description
to sign up for free,
because the first 2,000 people get 20% off
their annual membership.
So thanks to Brilliant for making learning fun,
thanks to Chopstix for not catching fire
and burning my place down,
and of course, thanks to you for watching.
(birds chirping)