- [Host] Robots with artificial intelligence
are already here.
Up until now they've mostly been focused on
assisting humans with well defined tasks.
But what about artificial intelligence
that can come up with its own ideas?
What about AI creativity?
Well that's what Gil Weinberg is exploring
in his lab at Georgia Tech.
- We really want to capture something
universal about music and emotions.
- [Host] This is Shimon, a robot that uses
artificial intelligence to write and perform
its own musical compositions.
Shimon represents an interesting exploration
of the future of artificial intelligence.
How can AI tools be leveraged to create?
This also begs certain questions like
what is creativity?
Can it be produced by robots?
Also, can robots battle rap?
Apparently the answer to that one is yes.
We're about to battle rap a robot.
What's up?
♪gimme all your money, ain't nothin' funny ♪
♪wanna get shot, drop ♪
Gil Weinberg has dedicated his career
to making human-robot music collaborations a reality.
- Music for me is the most amazing experience.
The holy grail is to create new experiences
that will give me this shiver down the spine approach.
Me and my listeners.
- [Host] He's created several musical robots
and AI controlled music devices like this.
This is Jason Barnes, the world's first bionic drummer.
His drumming arm was designed by Gil.
- AI for music and robotics, will allow people
like Jason Barnes, who is an amputee,
almost lost his passion to do anything, to be a musician.
- [Host] But with Shimon, Gil is trying something different,
a robot that creates its own music.
- Shimon is a robotic musician.
What I mean by musician is that it understands music
and it doesn't follow instructions,
it creates its own ideas.
- [Host] Shimon actually writes songs.
To accomplish this, Gil and his team
trained Shimon on a data set of around
40,000 progressive rock and jazz songs.
- [Gil] We started with Monk and Coltrane.
- [Host] They also trained Shimon
on music theory and what's known as rule based AI.
- [Gil] So rule based would mean go to a jazz textbook
and see how they teach in textbooks,
jazz improvisers to improvise.
- [Host] But the real magic is in
how Shimon adds his own spin on things.
- [Gil] It uses genetic algorithms
and fractiles and machine learning
and all kinds of processes that humans
will never use when they are creating music.
And here the promise is that it will create something novel
that will be surprising and inspiring.
That's how it pushes the envelope.
(xylophone music)
- [Host] All of this work led to Shimon v.1,
a robot that understands the rules of music composition,
can listen to human performers, and respond to them
with its own improvisations in real time.
(jazz funk music)
- The latest version of Shimon
takes things to the next level
by adding a singing capability
and integrating features that capture
human emotion and expression.
That's pretty crazy.
♪ oh I'm calling for you ♪
- [Host] I asked the team if they think
of Shimon as a machine or as an individual.
- I think of him ore as a robot when he's not working for me
if I'm like angry at him I think,
"oh this machine, how dare it!"
But when he's working it kind of feels like,
I don't know, sort of a buddy that you have
who's playing music for you, so yeah.
- [Host] Lisa's work is focused
on Shimon's head movements, his expression.
She's a PHD student and helps lead the project
to figure out what kind of movements
Shimon should do based on what he's playing.
How his eyebrows should move,
how to make Shimon convey that
he's kind of feeling what he's listening to or playing.
- When it detects a tempo, it will bob the head
with the tempo and immediately you feel
that it's grooving with you.
It's a completely different sense.
(jazz funk music)
- [Host] Shimon also writes his own lyrics.
- For lyrics, we use deep learning.
I gave Shimon's thinking, I gave him
a couple of priming words.
For example, in one song it was space and time.
I try to keep it abstract.
And then it went and generated
a bunch of lyrics around the notion of space and time.
- [Host] Gil's team then sends those lyrics
with the melody Shimon has written
to another team in Barcelona that
synthesizes Shimon's voice.
- Shimon's voice, as you hear it, is completely new
and they trained it on many voice samples.
♪ I must say my whole life ♪
♪ Shades of delight, rich as a book before night ♪
- [Host] If you ask me,
Shimon's singing is pretty impressive.
But singing isn't the only thing Shimon does,
Shimon also raps.
But not just any kind of rap, Shimon is a battle rapper.
I'm sure I'm not the only person who grew up on hip hop
who's fantasized about being in a rap battle.
You know, like that Eminem movie 8 Mile.
I finally got my chance.
Well, sort of.
- And we're ready to go.
♪ - I'm fighting for my time with a mathematical crime ♪
♪ aeronautical, cynical, sublime ♪
♪ the clock ticks on my rhyme. ♪
♪ - The clock has unwind your rhyme ♪
♪ You gotta do it double time, Shimon ♪
♪ Always on time, not in your head ♪
If you couldn't already tell,
neither Shimon or I are professional battle rappers.
But for Shimon there's some hope.
♪ - You're trying to make a claim ♪
♪ I want to be a human ♪
- [Host] This is the very first version
of Shimon raps, v1.0.
Rapping with Shimon did feel like we were collaborating.
I felt connected to Shimon, maybe not as a human
but not exactly a cold, heartless robot either.
It was pretty cool.
♪ - A city of rap distortion ♪
♪ Don't ask a first dangerous conversations ♪
- [Host] There's an old saying
that there's nothing new under the sun.
Everything that's new is just some
repurposing or new combination of old ideas.
In my experience, the most creative people
are the ones who consume the most ideas
and are best at recalling and synthesizing them
in an interesting way.
That's basically what Shimon does.
And in the future, robots that can consume ideas
will likely also get really good at synthesizing them
in ways that resonate with and surprise humans.
Shimon is a working first step.
I don't know if he, or it, will be entering
professional rap battle circuits any time soon.
But until then, let's enjoy the music.
(logo tone)
- Someone throw me a word and I'll try to work it in.
♪ Robot, cobot, I need mo bots in my life. ♪
♪ Like and subscribe, we might give you a bribe ♪
♪ In the form of a sweatshirt. ♪
♪ I'll rhyme until my head hurts. ♪
♪ Press that follow button. ♪
That's not gonna work (scoffs).
(laughter)
- Ta-da.