YUFENG GUO: Previously on AI Adventures,
we looked at how to use feature activations for understanding
neural networks.
How can we extend these ideas further and get even more out
of these methods?
Stay tuned to find out.
[MUSIC PLAYING]
Welcome to AI Adventures where we explore the art, science,
and tools of machine learning.
My name is Yufeng Guo.
And on this episode, we're going to extend
the ideas of feature activations in convolutional neural
networks even further by applying them to entire images
to create activation grids and then see
how we can use that to produce an activation atlas.
This video picks up where the previous video left off.
So if you haven't watched that, you should definitely
check it out first.
Now previously, we looked at how the neurons
in a convolutional neural networks layers
can be organized into channels and then stacked
like a layer cake.
We can also organize it in our mind vertically
so that we are looking at a certain patch of that layer
and consider the activation of all neurons stacked together
over a region.
We call this spatial activations.
Spatial activations give us a sense
of what features a specific region of an image
is activating most strongly.
By doing this across all the different patches of an image,
we can build an activation grid, showing us
the features that are activating the strongest for each region.
This is for only one particular layer.
So now we want to repeat this process for the other layers--
some earlier in the network and some later.
Earlier layers, like mixed 3a, they
show us the edges and patterns that still mildly resemble
the original image, while later layers
are much more identifiable as the animal body
parts that are being recognized in that part of image.
Now, activations are-- they're just numbers, right?
And so some activations, they can be larger than others.
So what we can now do is scale the size of each of these grids
to the size of the activation.
So that'll give us a sense of the importance of a given
region of the image to that layer based on how strongly
the network is activating.
So now we can see which regions are most strongly activated
per layer, and how these activations are propagating
through each layer of the network.
Now, this gives us a sense of the saliency
of the different parts of the image.
Activation grids tell us about some
of the activations of the neural network for a specific image.
But what about all the other activations of the network?
How might we understand the other types of activations?
Well, we'd need to feed in lots of example
images of all sorts of types.
So what if we did that?
We could run lots of images.
In fact, we could run a million different example images
through the network and record all of their activation routes.
All these individual activations correspond
to some kind of higher-dimensional location
in the neural networks activation space, so to speak.
And if we position our activations, each
of these subgrids, into their respective locations
in this higher-dimensional space,
we now have for our 1 million example
images broken up and scattered across this landscape.
Unfortunately, high-dimensional spaces
are, well, they're tough for us to visualize.
So we can project this space down to two dimensions.
But there's still the problem having
just millions of activations that deal with array.
So that's rather unwieldy for visualizing.
So let's draw a grid over the whole 2D projection.
And we'll take for each grid the average
of the activations within it.
This won't cause us to lose too much detail since activations
that are close together, they're similar to one another.
So now we can finally resize each grid
based on a number of activations on our grid,
giving us a sense of how strongly the network responds
to that particular type of input.
Now, we can repeat this entire process
for a couple different layers, put it
into a nice web interface, and you have an activation atlas.
So let's explore it and see what it
looks like to navigate through this space.
If we start our journey in the top area of the layer mixed 4c,
there appears to be kind of a variety of dog faces.
Then as we go down, things shift to showing how
the backs of four-legged animals.
And then we keep going.
We get to some legs and feet merging into the ground
and ground-related things.
And that slowly transitions into kind of sandy services
and some water.
There's lots of interesting pats through the activation atlas.
And it's really neat, I think, that the images shown,
they smoothly transition from one region to the next.
We can also look at a specific class activations in the atlas
so we can filter it out and just look at one particular class.
Let's say the great white shark.
I like to start with the bottom-most layer, mixed 5b,
since that's going to show images that
are most similar to kind of the final prediction
so we can really recognize what it is
we're seeing-- it's meaningful.
We see some regions we'd expect--
various fish-like imagery.
And over here there's some water-related imagery, right?
And what's this over here?
Yeah, is that a baseball?
What is a baseball doing so late in the network structure?
So if the network thinks that baseballs have a pretty strong
relationship to, well, great white sharks,
what would happen then?
Perhaps if we added an image of a baseball,
just slapped it on top of this image of a gray whale,
could we fool our neural network into thinking that it's
seeing a great white shark?
At first, it doesn't quite work.
But with a bit of fiddling around with the image size,
we can get the network to output great white shark
as its top prediction for the image.
This is pretty fun.
And you can look for all sorts of other oddities.
I also found in the mixed 5b layer for a great white shark
that airship is another kind of unusual label associated
with great white sharks.
So I tried to do something similar.
I went online and found some images of airships
and tinkered around with the sizing and position
of the airship image.
And I found that I could eventually convince the model
that it was a great white or in some cases, a killer
whale, depending on the airship I use and where I put it.
So you should try this out and be
sure to share what kinds of funny predictions
you got the model to make in the comments down below.
Neural networks are generally opaque.
But with tools like activation atlas at your disposal,
you can get a better understanding
of how they work and expose some of the weaknesses that
might be present.
I'm excited about the possibility
of how these approaches will be extended in the future, as well
as its applications to other types of models as well.
Thanks for watching this episode of Cloud AI Adventures.
And if you enjoyed it, please like
and subscribe to get all the latest episodes right
when they come out.
For now, check out activation atlases
and see if you can find something
interesting in the relationships between the different data
classes.
[MUSIC PLAYING]