Say anytime you're talking to your phone--
that's pretty much a neural net doing that.
It does really well on image recognition,
autonomous cars coming soon,
the top flight method for genetic sequencing
is a neural network.
You can say they're loosely inspired by the brain.
It is a network of neurons, people call it neurons.
They're all connected to each other
going down through the layers.
In a way they mimic neurons in that
they're little triggers.
Each neuron has kind of a threshold--
called a weight where it makes a decision.
So it takes in data, it could be huge amount images.
And then once the network is trained--
You can put one image in at the front.
They fire when they see the thing that they want to see.
So you get this kind of cascade
moving through the neural network.
And that does really poorly. it just does horribly.
But there's this little magic trick
called back propagation and this is very unbiological--
Your neurons don't do this at all.
But at the end you're like: "OK, well you were wrong.
But here's a proper picture of a dog."
And now you send all that information
back through the network again
and the network learns just a little bit
better at how to say: "Oh this is a dog."
It's elegant math and really the breakthroughs
in the field were when they stopped
trying to be so biological in the 1980's.
At the time they didn't work that great
but it turned out all they needed
was a huge amount of processing power
and a huge amount of examples
and they would really sing.
So in many of these cases accuracy is enough
but when you're talking really life and death decisions--
driving autonomous cars, making medical diagnoses,
or pushing the frontiers of science
to understand why a discovery was made--
you really need to know what the AI is thinking.
Not just what its results are.
Since you have all these layers it just engages
in such complex decision-making
that it just is essentially a black box.
We don't really understand how they think.
All is not lost with this black box problem.
People are actually trying to solve this now
through a variety of different ways.
One researcher has created a toolkit to get at
the activation of individual neurons in a neural network.
This toolkit works by taking this individual neuron
and telling the network to go back
and find all the weights that really make that neuron
just fire like crazy.
Keep doing this thousands of times and you can see
the kind of perfect input for this neuron.
Then looking inside these layers he could see
that some neurons learn these really complex,
abstract ideas like a face detector--
something that can detect any human face,
no matter what it looks like.
This is not a property you would expect
an individual neuron to have.
Most don't, but some do.
Many neural net researchers think of it this way:
You can think of the decision making of a neural net
as this kind of terrain of valleys and peaks
and the ball represents this piece of data.
So you can understand this one decision
that was made in one valley,
but all those other valleys
you have no idea what's going on.
So one way to get at what an AI is thinking
is to find a proxy for what it's thinking.
So one professor has taken the video game Frogger
and trained an AI to play the game extremely well--
because that's fairly easy to do now.
But, you know, what is it deciding to do?
It's really hard to know
especially in a sequence of decisions
and a dynamic environment of that.
So rather than trying to get the AI itself to explain itself
He asked people to play the video game
and have them say what they were doing
as they were playing it
and recorded the state of the frog at the same time.
And then he found a way to use neural networks
to translate those two languages:
the code of the game and what they were saying
And then he imported that back
to the game playing network.
So then the network was armed with these
human insights.
So, as the frog is waiting for a car to go by
It'll say: "Oh, I'm waiting for a hole to open up
before I go."
Or it could get stuck on the side of the screen
and it would say: "Uh geez, this is really hard and curse.
So it's kind of lashing human decision-making
on to this network with more deep networks.
It's all about trust.
If you have a result
and you don't understand why it made that decision--
how can you really advance in your research?
You really need to know that there's not
some spurious detail that's throwing things all off.
But these models are just getting larger
and more complex and I don't think anyone thinks
they'll get to a global understanding
of what a neural network is thinking anytime soon.
But if we can get a sliver of this understanding--
science can really push forward
and these neural networks can really play.