[ambient electronic music fades in]
[voice of male] Every day I wake up,
and I kind of think to myself, how did I get here?
[voice of announcer over drumming and cheering] Jason Barnes. Jason Barnes, come out here. [Jason drumming]
[voice of Jason] But, you put your mind to something,
and you don’t let anything stop you,
it’s impossible to not make progress.
[fast-paced drumming]
[upbeat electronic music begins]
[voice of female] When Jason was in diapers,
I knew he was going to be an unusual child.
He would be in his high chair at the dinner table,
and if there was any music on, he would be beating on the table.
Tap, tap, tap. Tap, tap, tap.
We realized, this kid needs some drums,
because he’s always beating up everything.
And he had a band, and then life changed.
[music fades out] And he had a band, and then life changed.
January 9th, 2012. I’ll never forget.
[voice of Jason] I used to do subcontract work, cleaning restaurant exhaust.
We’d usually do like 2 to 3 jobs a day.
Go up on the roof, spray down the duct.
Just like I remember a pink flash and like a loud boom.
[ambulance siren]
And then I woke up in the hospital.
[voice of Maggi] When I knew that we were going to have to amputate his arm,
he cried in my arms.
He said, “Mom, I’m never going to amount to anything.
I’ll never play drums.
I’ll never be able to do anything again in my life, ever.”
After 3 weeks into him being home,
he said, “I can’t take this anymore.”
He said, “Get my drum kit out of the garage.”
And, so, we duct taped a drumstick to his arm.
He did some little tapping around, and then he turned around and said,
“You know, I think I can make something work.”
Does it seem like it’s growing back, Doc?
[laughs]
[voice of Jason] There’s a side of me,
ever since I’ve been a kid where if I’m going to start something,
it has to get done.
I just won’t let it go.
So, my current prosthetic is what you’d call a body-powered prosthetic.
It just uses basic mechanics,
so there’s no robotic controls or no electronics.
[drumming] I use a lot of my shoulder and elbow to kind of exaggerate,
[drumming] to get the hits that I want.
It’s pretty exhausting. It also has limitations on how fast I can play.
At first it was a little slow.
Yeah, it’s a weird one.
So, I had an idea of developing a robotic drumming prosthetic,
and Gil Weinberg develops musical robots.
And, so, I knew Gil was kind of the right person to reach out to.
[voice of technician] Ready?
[voice of Gil] I understood it immediately, what he wanted.
He wanted to sense more of his muscles
and be able to control the bounce of the drumstick.
[voice of Jason] Basically, we just wanted to
be able to get me back on the kit
and play as humanly natural as possible.
[voice of Gil] The model is trying to learn you,
but you probably will have to learn how to work with the model.
[voice of Jason] I think that most of it now is the sensitivity.
I’m not training it that hard.
[voice of Raghavasimhan] Also there is a trade-off, because …
If you increase the sensitivity,
then it’s going to have a lot of
false positives and false negatives.
[voice of Gil] Would you like to move stiff, loose?
How fast are we going to change between the kind of hits?
[voice of Jason] I mean I have tons of ideas.
And, then it’s just like, how accurate is everything going to be?
— You know what I mean? — Mmhmm.
[voice of Gil] I was a little worried.
Can we provide him with what he wanted
in terms of the latency and the control.
We didn’t know that it’s possible.
OK, I’m really excited to see if we have signals.
— Ready, Ning?
The goal is very simple,
but in reality, in each of the steps,
it’s actually pretty complicated.
Oh, see? That’s what happened with me last time.
[voice of Gil] We have some crashes.
And, the stick starts to roll around.
Maybe we should have Raghav listening also to get his insight.
[voice of Jason] Like we were doing in the lab.
— Got you.
[voice of Ning] It could be also the power supply itself.
— Right.
[voice of male] There are actually a handful of challenges
with creating a prosthetic arm,
specifically for drumming.
[tentative tapping]
[tentative tapping] Latency is always a big concern.
[tentative tapping] You don’t want to flex your arm
and a second later for it to do something.
— Yeah.
[voice of Nat] And, there’s a lot of nuance
with exactly how the drumstick reacts:
things as subtle as how tightly you grip it,
how does it bounce.
They’re really hard to calibrate
[drum tapping] if you don’t have a machine learning system
[drum tapping] to gather all the data.
[voice of woman] When I heard about Jason’s story,
it really blew my mind.
I was, like, whoa.
I had no idea how
far we have come
in terms of AI and prosthetics.
[voice of Jason] So, the way the arm operates
is we use EMG,
which stands for electromyography.
Basically what that is,
is it’s sensors that are amplified
by a battery that pick up electrical signals
from my residual limb.
[voice of Sarah] Using these signals,
we are able to do recognition of complex patterns
with machine learning
and then map them to specific movements.
TensorFlow Lite is doing that in real time.
[voice of Jason] And, so, I can flex my muscle
and it will tighten the grip on the stick.
And, I can extend my muscle,
and it will loosen the grip on the stick,
just like it would with a normal hand.
No, the latency, it’s like none.
[voice of Gil] Awesome. That’s awesome.
[voice of Gil] And, in the last few months,
we were able to detect,
almost 100% of the time, his intention.
[drumming] [voice of Jason] I can actually feel the feedback from the arm,
and it feels as close to a real hand as you can get
without it actually being a real hand.
[voice of Sarah] I think what Jason and the team
[drumming] have accomplished is truly awe-inspiring.
He took what was a large setback in his life,
and he moved an entire field of research forward.
So loose, you can actually …
[fast tapping on cymbals]
[voice of male] Let me see what it might even ….
[cymbals]
Yeah, there are drummers with two sound hands
that don’t have double strokes that even.
[laughing] that don’t have double strokes that even.
[laughing] It’s amazing.
[voice of Gil] For the first time, if you look at him playing,
you may not even notice this technology.
You just see Jason as the great
and expressive musician that he is.
[fast, energetic drumming]
[electronic music joins in]
[voice of Jason] Of course I think there’s a need to keep pushing on this stuff,
because there’s always someone
that needs help with something like this.
And, AI is going to be the bridge between that.
[rock music fades out]