Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
In the previous episode, we talked about image classification, which means that we have an
image as an input, and we ask a computer to figure out what is seen in this image.
Learning algorithms, such as convolutional neural networks are amazing at it, however,
we just found out that even though their results are excellent, it is still quite hard to find
out how they get to make a decision that an image depicts a dog or cat.
This is in contrast to and old and simple technique that goes by the name bag of words.
It works a bit like looking for keywords in a document and by using those, trying to find
out what the writing is about.
Kind of like the shortcut students like to take for mandatory readings.
We have all done it.
Now, imagine the same for images, where we slice up the image into small pieces and keep
a score on what it seen in these snippets.
Floppy ears, black snout, fur, okay, we’re good, we can conclude that we have a dog over
here.
But wait, I hear you are saying, Károly, why do we need to digress from AI to bag of
words?
Why talk about this old method?
Well, let’s look at the advantages and disadvantages and you will see in a moment.
The advantage of bag of features is that it is quite easy to interpret because it is an
open book: it gives us the scores for all of these small snippets.
We know exactly how a decision is being made.
A disadvantage, one would say is that because it works per snippet, it ignores the bigger
spatial relationships in an image, and therefore overall, it must be vastly inferior to a neural
network, right?
Well, let’s set up an experiment and see!
This is a paper from the same group as the previous episode at the University of Tübingen.
The experiment works the following way: let’s try to combine bag of words with neural networks
by slicing up the image into the same patches, and then, feed them into a neural network
and ask it to classify them.
In this case, the neural network will do many small classification tasks on image snippets
instead of one big decision for the full image.
The paper discusses that the final classification also involves evaluating heatmaps and more.
This way, we are hoping that we get a technique where a neural network would explain its decisions,
much like how bag of words works.
For now, let’s call these networks BagNets.
And now, hold on to your papers, because results are really surprising.
As expected, it is true that looking at small snippets of the image can lead to misunderstandings.
For instance, this image contains a soccer ball, but when zooming into small patches,
it might seem like this is a cowboy hat on top of the head of this child.
However, what is unexpected is that even with this, BagNet produces surprisingly similar
results to a state-of-the-art neural network by the name ResNet.
This is…wow.
This has several corollaries.
Let’s start with the cool one: this means that neural networks are great at identifying
objects in scrambled images, but humans are not.
The reason for that is that the order of the tiles don’t really matter.
We now have a better reason why this is the case — doing all this classification for
many small tiles independently has superpowers when it comes to processing scrambled images.
The other, more controversial corollary is that this inevitably means that some results
that show the superiority of deep neural networks over the good old bag of features come not
from using a superior method, but from careful fine-tuning.
Not all results, …some results.
As always, a good piece of research challenges our underlying assumptions, and sometimes,
in this case, even our sanity.
There is a lot to say about this topic and we have only scratched the surface, so take
this as a thought-provoking idea that is worthy of further discussion.
Really cool work, I love it.
This video has been supported by Audible.
By using Audible, you get two excellent audiobooks free of charge.
I recommend that you click the link in the video description, sign up for free and check
out the book Superintelligence by Nick Bostrom.
Some more AI for you whenever you are stuck in traffic, or have to clean the house.
I talked about this book earlier and I see that many of you Fellow Scholars have been
enjoying it.
If you haven’t read it, make sure to sign up now because this book discusses how it
could be possible to build a superintelligent AI, and what such an all-knowing being would
be like.
You get this book free of charge, and you can cancel at any time.
You can’t go wrong with this.
Head on to the video description and sign up under the appropriate links.
Thanks for watching and for your generous support, and I'll see you next time!