(gentle music)
- What do you see in these pictures?
Objects, faces, or do they just look
like random arrangements of lines and blobs?
If you don't see anything specific it's probably
because you're not a computer.
These pictures were specially designed
to be recognized by machine vision algorithms.
You see blobs, they see a shark,
binoculars, explicit nudity.
But the algorithms that see these things
are the same ones being used
in more and more parts of everyday life.
They steer self-driving cars, they monitor content
on the internet and they even unlock your phone.
And the fact that they don't see the world the same way
you do it could be a problem.
(cosmic music)
The science of teaching computers to see goes back
to the 1960s and it coincides with the creation
of the field of artificial intelligence.
Early computer vision systems were very basic.
They could process only the simplest versions of 3D scenes
rendering the world in crude shapes and planes.
But in recent years a revolution in AI,
particularly deep learning, has created sophisticated
vision systems which can outperform humans
at a number of tasks.
To date we've used deep learning to create vision
algorithms that can identify cancerous tumors
better than a doctor or tell the difference
between a hundred different dog breeds in a millisecond.
Or they can just tell you whether the food
on your plate is a hotdog or not.
Okay, most humans can do that too.
But for all these achievements AI doesn't look
at the world the same way humans do
and that's what these bizarre prints
are meant to demonstrate.
They're the work of Tom White, an academic
and an artist from New Zealand.
He made them by essentially reverse engineering
a number of vision systems like those used
by Google and Amazon.
- So I started looking at these classifier networks
which knew how to classify or how to understand
an image and I was wondering if I can invert that process.
- [James] The prints are generated using a production line
of algorithmic programs.
First, a drawing system generates some abstract lines
and the image is fed into a machine vision classifier
which then tries to guess what object it might be.
Based on the classifier's guesses the drawing system
then tweaks the image and feeds it through again.
- I mostly take a hands-off approach because I really want
to know how the algorithms see the world.
And so after I set the systems up I kind of sit back
and let it run for a long time and see what comes out.
- [James] But if that's an algorithm's view
what does it look like to humans?
We asked Verge staff to guess which object
each print represented.
In essence we asked people can you think like a computer?
- Okay, you want me to say what I think that is?
- Like I'm just imagining like it's under water
and they're a bunch of fish or bugs.
- This one just looks like hot topic.
- A car?
- [Tom] It's a Starfish.
- It's a starfish, great.
- A train on train tracks, subway.
- This is someone getting pushed in front of a train.
- It definitely looks like somebody's conducting
with the wave motions.
- Like something on a stove?
- [Tom] It's a spider.
- Oh, for real?
- (chuckles) Okay.
- This is a dancing elephant.
- Like the wheel of a boat.
- An elephant.
- This was a whale.
Is that a whale?
Did people do good on this?
Like am I the, okay.
- Tom's work plays with the limits of AI
but in an abstract way.
So here's a more concrete example.
Let's say you're a computer scientist
and you're training a machine learning program
to recognize animals at the zoo.
You would collect a bunch of pictures of zebras, lions,
giraffes, et cetera, and you'd feed them into this algorithm
which would then look for patters in this data.
So after studying the pictures it might conclude
that if it sees something with a long neck, boom, giraffe.
If it sees stripes that's a zebra and so on.
There are problems with this approach though.
First, the algorithm doesn't know anything
about the world beyond that data.
So if you get a new edition to the zoo,
like a penguin, it's not gonna know what that is.
The second problem is that the way it makes decisions
might not be very sensible.
If it decides that all animals with stripes are zebras,
for example, what happens when it sees a tiger?
This is why researchers often describe AI
as narrow or brittle.
These systems only work in very limited scenarios
and when they come across unexpected data
they often break down.
This becomes really clear when you look back at Tom's art.
For example, look at this red print here (cello music).
The object it's supposed to represent is a cello.
You can see the curves of the body of the instrument
and the vertical lines with strings.
But there's also this other shape hovering
behind it in light red.
That's the person playing the cello.
Now the reason this figure appears is because the pictures
that we used to train the algorithm
included the cellos and the cellists too.
But because the program has no understanding
of what an instrument or a musician is
it just blurs the two together.
Or there's Tom's tricycle.
- [Tom] When I see a tricycle the first thing I think
about are the wheels and maybe in the back
of my head I count them.
- [James] But vision algorithms are terrible at counting
so the number of wheels is no help.
They lock onto the shape of the frame, a triangle,
and think that that represents the essence
of a tricycle at its best.
- To me this shape that it comes up with
doesn't remind me of a tricycle.
It looks like a bunch of lines but I guess I gain
an appreciation for the different ways of viewing things,
It's almost like visiting another culture
that has different ways of interpreting
or relating to objects.
- [James] To drive home the differences between human
and machine vision we decided to turn the tables
on the algorithms.
So we took the same objects in Tom's art
and abstracted them using a human algorithm.
We made the algorithm play Pictionary against us.
- So I'm drawing a spider, weirdly kind of intimidates me.
See what the computer thinks.
Well they said it was an invertebrate but I actually
can't remember now if spiders are invertebrates or not.
- The cello says text, black and white drawing, wing, joint.
The cellos are more complex so it didn't get that at all,
like anywhere near that.
- But why does this matter?
Well because despite the limitations of machine vision
we're trusting it with more and more aspects of our lives.
Take self-driving cars, for example.
In the future, the idea is that they'll rely totally
on what computers can and cannot see, no humans needed.
So teaching a machine to spot the difference
between a pedestrian and a stop sign will literally
be a life or death matter.
The first fatal crash involving Tesla's self-driving
self-drive, for example, was partly caused
by algorithms which couldn't distinguish
between the side of a white tractor trailer
and the bright sky behind it.
And when you think about the other places we're starting
to use these algorithms in CCTV cameras,
in military drones, it becomes worrying.
Now this doesn't mean we're building
completely broken systems.
Machine learning engineers are aware of these shortcomings
and most algorithms, like the ones we described,
still have humans in the loop somewhere making decisions.
And Tom, he takes solace in the fact that the algorithms
aren't any smarter than this.
He suggests that we should be thankful that a computer
still struggles to count the numbers
of wheels on a tricycle.
- I think it's kind of refreshing to see
even when they have very simple models of the world,
in a way that's comforting.
It's good to know how these things work.
It sometimes can give us insight into different ways
that we can see the world.
- [Lady] In this one I have birds, like it's a bird.
It's a bird maybe being crushed in someone's fist.
It's like a bird.
- [Tom] It's a butterfly.
- It's a but, yeah okay.
- [Tom] I like the stories. - I see it.
- [Tom] A bird being crushed in someone's fist?