Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
Hold on to your papers because this is an exclusive look at a new neural network visualization
paper that came from a collaboration between Google and the Carnegie Mellon University.
The paper is as fresh as it gets because this is the first time I have been given an exclusive
look before the paper came out, and this means that this video and the paper itself will
be published at the same time.
This is really cool and it's quite an honor, thank you very much!
Neural networks are powerful learning-based tools that are super useful for tasks that
are difficult to explain but easy to demonstrate.
For instance, it is hard to mathematically define what a traffic sign is, but we have
plenty of photographs of them.
So the idea is simple, we label a bunch of photographs with additional data that says
"this is a traffic sign", and "this one isn't", and feed this to the learning algorithm.
As a result, neural networks have been able to perform traffic sign detection at a superhuman
level for many years now.
Scientists at Google DeepMind have also shown us that if we combine a neural network with
reinforcement learning, we can get it to look at a screen and play computer games on a very
high level.
It is incredible to see problems that seemed impossible for many decades crumble one by
one in quick succession over the last few years.
However, we have a problem, and that problem is interpretability.
There is no doubt that these neural networks are efficient, however, they cannot explain
their decisions to us, at least not in a way that we can interpret.
To alleviate this, earlier works tried to visualize these networks on the level of neurons,
particularly, what kinds of inputs make these individual neurons extremely excited.
This paper is about combining previously known techniques to unlock more powerful ways to
visualize these networks.
For instance, we can combine the individual neuron visualization with class attributions.
This offers a better way of understanding how a neural network decides whether a photo
depicts a labrador or a tiger cat.
Here, we can see which part of the image activates a given neuron and what the neuron is looking
for.
Below, we see the final decision as to which class this image should belong to.
This next visualization technique shows us which set of detectors contributed to a final
decision, and how much they contributed exactly.
Another way towards better interpretability is to decrease the overwhelming number of
neurons into smaller groups with more semantic meaning.
This process is referred to as factorization or neuron grouping in the paper.
If we do this, we can obtain highly descriptive labels that we can endow with intuitive meanings.
For instance, here, we see that in order for the network to classify the image as a labrador,
it needs to see a combination of floppy ears, doggy forehead, doggy mouth, and a bunch of
fur.
And we can also construct a nice activation map to show which part of the image makes
our groups excited.
Please note that we have only scratched the surface.
This is a beautiful paper, and it has tons of more results available exactly from this
moment, with plenty of interactive examples you can play with.
Not only that, but the code is open sourced, so you are also able to reproduce these visualizations
with little to no setup.
Make sure to have a look at it in the video description.
Thanks for watching and for your generous support, and I'll see you next time!