Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
This paper is not your usual paper, but it does something quite novel.
It appeared in the Distill journal, one of my favorites, which offers new and exciting
ways of publishing beautiful, but unusual works aiming for exceptional clarity and readability.
And of course, this new paper is no different.
It claims that despite the fact that these neural network-based learning algorithms look
almost unfathomably complex inside, if we look under the hood, we can often find meaningful
algorithms in there.
Well, I am quite excited for this, so, sign me up!
Let’s have a look at an example!
At the risk of oversimplifying the explanation, we can say that a neural network is given
as a collection of neurons and connections.
If you look here, you see the visualization of three neurons.
At first glance, they look like an absolute mess, don’t they?
Well, kind of, but upon closer inspection, we see that there is quite a bit of structure
here!
For instance, the upper part looks like a car window, the next one resembles a car body,
and the bottom of the third neuron clearly contains a wheel detector.
However, no car looks exactly like these neurons, so what does the network do with all this?
Well, in the next layer, the neurons arise as a combination of neurons in the previous
layers, where we cherry pick parts of each neuron that we wish to use.
So here, with red, you see that we are exciting the upper part of this neuron to get the window,
use roughly the entirety of the middle one, and use the bottom part of the third one to
assemble…this!
And now, we have a neuron in the next layer that will help us detect whether we see a
car in an image or not.
So cool!
I love this one!
Let’s look at another example.
Here you see a dog head detector, but it kind of looks like a crazy Picasso painting where
he tried to paint a human from not one angle like everyone else, but from all possible
angles on one image.
But this is a neural network, so why engage in this kind of insanity?
Well, if we have a picture of a dog, the orientation of the head of the dog can be anything.
It can be a frontal image, look from the left to right, right to left, and so on.
So this, is a pose invariant dog head detector!
What this means is that it can detect many different orientations, and look here!
You see that it gets very excited by all of these good boys.
I think we even have a squirrel in here.
Good thing this is not the only neuron we have in the network to make a decision!
I hope that it already shows that this is truly an ingenious design.
If you have a look at the paper in the video description, which you should absolutely do,
you’ll see exactly how these neurons are built from the neurons in the previous layers.
The article contains way, way more than this, you’ll see a lot more dog snouts, curve
detectors, and even a followup article that you can have a look at and even comment on
before it gets finished!
A huge thank you to Chris Olah, who devotes his time away from research and uses his own
money to run this amazing journal, I cannot wait to cover more of these articles in future
episodes, so make sure to subscribe and hit the bell icon to never miss any of those.
So, finally, we understand a little more how neural networks do all the amazing things
they are able to do.
What a time to be alive!
This episode has been supported by Weights & Biases.
Here, they show you how you can visualize the training process for your boosted trees
with XGBoost using their tool.
If you have a closer look, you’ll see that all you need is one line of code.
Weights & Biases provides tools to track your experiments in your deep learning projects.
Their system is designed to save you a ton of time and money, and it is actively used
in projects at prestigious labs, such as OpenAI, Toyota Research, GitHub, and more.
And, the best part is that if you are an academic or have an open source project, you can use
their tools for free.
It really is as good as it gets.
Make sure to visit them through wandb.com/papers or just click the link in the video description
and you can get a free demo today.
Our thanks to Weights & Biases for their long-standing support and for helping us make better videos
for you.
Thanks for watching and for your generous support, and I'll see you next time!