Translator: Tingting Zhong Reviewer: lisa thompson
I'm Pindar.
I like to paint,
and I like to write software.
So, about 10 years ago,
I built a painting robot.
It was a robot that I built to paint like I did, with a brush on canvas.
I've built four or more robots since,
each of them with more sophisticated artificial intelligence,
each of them trying to paint more like I painted.
We can paint in a lot of different styles
though recently we've been concentrating on portraits,
like this one here, of my children.
These robots, they surpassed my technical ability a long time ago,
and creatively, they're catching up.
To give you guys an idea of just how creative they have become
and what I even mean by creativity,
I am going to first tell you about a bunch of the algorithms they use,
and then I am going to tell you in detail about one algorithm,
or one process, called a feedback loop.
Then I'll walk you through the creation of a couple of paintings.
I'm going to let you decide for yourself just what you think is going on here,
decide for yourself if you think these machines are actually creative.
But this is all easier to show than explain,
so let's look at some of these portraits.
This is the very first portrait of my son.
What I did is I would feed the computer dots,
and it would just connect them.
It was connecting the dots, not that creative at all.
Over the next several years, however,
I started experimenting with various artificial intelligence algorithms,
and with each of them, I tried to make the robots more independent.
These algorithms included algorithms to make their own dots to connect,
sometimes simple, like in this, when I repeated that other portrait,
sometimes more complex.
I had processes that would balance compositions;
I had others that would create unique compositions;
I had others that would try to paint stylistically.
My favorite one of all was called k-means clustering, which I'm using here
in the creation of a portrait of my daughter,
and I use that to create color palettes
and find ways to paint those color palettes.
About two years ago,
I opened up control of the robot to the internet
and to anyone to paint with it.
All you needed to do was go to a website and trace over an image,
and then the robot would immediately take your strokes
and trace what you were tracing.
A lot of people had a lot of fun with this.
And this led to a lot of crowd-sourced art into my current experimentation,
where I try and watch how people are using the robot,
and take direction from them and actually learn from them.
So, this is actually an interesting portrait
that has a little of everything.
It had a lot of the robot's algorithms,
hundreds of people from around the world were painting with it.
At the end, I gave my kids - I really love this -
I gave my kids the opportunity to do the final strokes,
and they seized that opportunity and played tic-tac-toe on Lincoln's face.
(Laughter)
I think you can see it right there - the eyes.
The "O" is right on his eyes.
So, the one algorithm that I wanted to tell you about in detail,
the process is called a feedback loop.
It sounds like something you would hear about in a computer science class,
but I learned about it in art school,
when I was reading about painter Paul Klee's take on creativity.
For him, every painting had a beginning and an end,
and in between,
the painter or the artist would switch back and forth between making a mark
and taking a step back to evaluate,
then making another mark and taking another step back to evaluate,
over and over again in this feedback loop.
And that really made a lot of sense to me because that's how I painted my portraits.
I decided that I wanted my robots to paint the same way.
But there was a problem, of course, and that is that my robots were blind;
they couldn't see what they were doing.
So way back between my second and third robot, I added a camera,
and I pointed that camera at the canvas.
So now, every robot since has been able to watch what it was painting.
I'm going to show you this time-lapse,
where you can see how the robot watches what it is painting
and makes adjustments compared to what it's doing in this time-lapse.
Let's rewind to the beginning, and you'll see -
all the brushes are getting taken off.
I'll play it forward in a second,
but you can see here what we started with -
just a blank canvas and a photograph.
And then here, I am going to put this little visual aid for you.
And this visual aid, this heat map in the middle
shows how the robot has calculated the differences
between the canvas and the photo.
The brighter the color, the greater the difference.
So, when I hit play on this time-lapse,
you will see the robot seek out the areas with the greatest difference,
look at the bottom,
find the color that will make the canvas most like the painting,
and paint that brushstroke.
Then it will take another picture to recalculate.
So watch as the painting progresses and the differences fade away.
A lot of people want to see that again.
I'm going to show you another one
so you can see something like that happen again.
The reason I want to move on
is because this painting has a lot more than just feedback loops going on in it.
Algorithms that were -
You can see I did a lot of brushstrokes with my own finger,
and that is how I typically like to make paintings with my robots;
I like to collaborate with them.
But it doesn't always have to be like that.
This next painting, which I made with my son,
was done completely by the robot.
In fact, my son and I,
we don't know if we like to say, or even can say we made it,
because few, if any, creative decisions were made by us.
The only decision we made was to sit down for a bunch of photos.
From there, the robot picked its favorite,
created an unique composition, and painted.
It did this by, first, using the Viola-Jones face detection algorithm
to find his face in each photo.
If it couldn't find a face, it would toss and disregard.
You can see the green square around his face.
It then used Haar cascades to find his eyes and mouth,
and then it used an algorithm we wrote
to look at the eyes and mouth and measure the symmetry, the proportion;
and calculate the direction he was looking in;
to pick a favorite;
and then to crop the image into an original composition.
From those original compositions, I used k-means clustering -
that algorithm I told you about earlier -
first to create an underpainting,
and then to create something for the more nuanced tones.
From there, the robot painted.
But not blindly.
The robot used its camera and feedback loops
to try and remove the differences
between the canvas and the original composition that it had come up with.
Something even more interesting
is that it used feedback loops to decide when to stop.
What happened was
as soon as it realized its strokes were not making the painting
any more like the photo that it was trying paint,
it just decided it had done its best, and it was over.
So, when you think about this,
when this robot started, it didn't know what it was going to paint.
With each brushstroke, it didn't know what the next brushstroke was going to be
until it considered all the previous brushstrokes
and the effect it had on the painting.
And it didn't even know when it was going to stop
until eventually, it looked at -
after one brushstroke, it looked at the painting
and thought to itself, "Wow, this is the best I can do."
And it was over.
The reason I find this very interesting
is because this is very similar to how I paint portraits.
I know I told you earlier I was going to let you decide -
and of course, everyone is free to decide
whether or not they think machines can be creative -
but I have no choice but to accept it now,
because if I don't accept that these machines are creative,
then it calls into question whether or not -
when I paint my portraits, whether or not I am being creative.
So, it's actually kind of sweet
when you compare the very first portrait to our most recent.
It is a coincidence, but it is true that both of them are of my second son.
It shows how far we've come
from when we first started connecting the dots
to painting portraits completely on our own.
I've gotten them this far by taking my own creative process
and breaking it down into pieces small enough
that I was able to teach it to a machine.
This is all about teaching -
teaching the most naive student imaginable
how to do something that comes easy to us,
trying to teach it how to be creative.
As we continue to work and as we continue to experiment
with new artificial intelligence algorithms,
I'm looking forward to the day that their creativity surpasses mine,
and with even greater anticipation,
I'm looking forward to the day
that they do something that I wasn't expecting
and something that really surprises me,
perhaps even by doing something they weren't taught.
Thank you very much.