Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
When it comes to image classification tasks, in which the input is a photograph and the
output is decision as to what is depicted in this photo, neural network-based learning
solutions became more accurate than any other computer program we, humans could possibly
write by hand.
Because of that, the question naturally arises: what do these neural networks really do inside
to make this happen?
This article explores new ways to visualize the inner workings of these networks, and
since it was published in the Distill journal, you can expect beautiful and interactive visualizations
that you can also play with if you have a look in the video description.
It is so good, I really hope that more modern journals like this appear in the near future.
But back to our topic - wait a second, we already had several videos on neural network
visualization before, so what is new here?
Well, let’s see!
First, we have looked at visualizations for individual neurons.
This can be done by starting from a noisy image and add slight modifications to it in
a way that makes a chosen neuron extremely excited.
This results in these beautiful colored patterns.
I absolutely love, love, love these patterns, however, this misses all the potential interactions
between the neurons, of which there are quite many.
With this, we have arrived to pairwise neuron activations, which sheds more light on how
these neurons work together.
Another one of those beautiful patterns.
This is, of course, somewhat more informative: intuitively, if visualizing individual neurons
was equivalent to looking at a sad little line, the pairwise interactions would be observing
2D slices in a space.
However, we are still not seeing too much from this space of activations, and the even
bigger issue is that this space is not our ordinary 3D space, but a high-dimensional
one.
Visualizing spatial activations gives us more information about these interactions between
not two, but more neurons, which brings us closer to a full-blown visualization, however,
this new Activation Atlas technique is able to provide us with even more extra knowledge.
How?
Well, you see here with the dots that it provides us a denser sampling of the most likely activations,
and, this leads to a more complete bigger-picture view of the inner workings of the neural network.
This is what it looks like if we run it on one image.
It also provides us with way more extra value, because so far, we have only seen how the
neural network reacts to one image, but this method can be extended to see its reaction
to not one, but one million images!
You can see an example of that here.
What’s more, it can also unveil weaknesses in the neural network.
For instance, have a look at this amazing example where the visualization uncovers that
we can make this neural network misclassify a grey whale for a great white shark, and
all we need to do is just brazenly put a baseball in this image.
It is not a beautiful montage, is it?
Well, that’s not a drawback, that’s exactly the point!
No finesse is required, and the network is still fooled by this poorly-edited adversarial
image.
We can also trace paths in this atlas which reveal how the neural network decides whether
one or multiple people are in an image, or how to tell a watery type terrain from a rocky
cliff.
Again, we have only scratched the surface here, and you can play with these visualizations
yourself, so make sure to have a closer look at the paper through the link in the video
description.
You won’t regret it.
Let me know in the comments section how it went!
Thanks for watching and for your generous support, and I'll see you next time!