Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
In the last few years, neural network-based learning algorithms became so good at image
recognition tasks that they can often rival, and sometimes even outperform humans in these
endeavors.
Beyond making these neural networks even more accurate in these tasks, interestingly, there
is also plenty of research work on how to attack and mislead these neural networks.
I think this area of research is extremely exciting and I’ll now try to show you why.
One of the first examples of an adversarial attack can be performed as follows.
We present such a classifier with an image of a bus, and it will successfully tell us
that yes, this is indeed a bus.
Nothing too crazy here.
Now, we show it not an image of a bus, but a bus plus some carefully crafted noise that
is barely perceptible, that forces the neural network to misclassify it as an ostrich.
I will stress that this is not any kind of noise, but the kind of noise that exploits
biases in the neural network, which is, by no means easy or trivial to craft.
However, if we succeed at that, this kind of adversarial attack can be pulled off on
many different kinds of images.
Everything that you see here on the right will be classified as an ostrich by the neural
network these noise patterns were crafted for.
In a later work, researchers of the Google Brain team found that we can not only coerce
the neural network into making some mistake, but we can even force it to make exactly the
kind of mistake we want!
This example here reprograms an image classifier to count the number of squares in our images.
However, interestingly, some adversarial attacks do not need carefully crafted noise, or any
tricks for that matter.
Did you know that many of them occur naturally in nature!
This new work contains a brutally hard dataset with such images that throw off even the best
neural image recognition systems.
Let’s have a look at an example.
If I were the neural network, I would look at this squirrel and claim that “with high
confidence, I can tell you that this is a sea lion”.
And you human, may think that this is a dragonfly, but you would be wrong.
I am pretty sure that this is a manhole cover!
Except that it’s not.
The paper shows many of these examples, some of which don’t really occur in my brain.
For instance, I don’t see this mushroom as a pretzel at all, but there was something
about that dragonfly that, upon a cursory look, may get registered as a manhole cover.
If you look quickly, you see a squirrel here…just kidding, it’s a bullfrog.
I feel that if I look at some of these with a fresh eye, sometimes I get a similar impression
as the neural network.
I’ll put up a bunch of more examples for you here, let me know in the comments which
are the ones that got you.
Very cool project, I love it.
What’s even better, this dataset by the name ImageNet-A is now available for everyone,
free of charge.
And if you remember, at the start of the video, I said that it is brutally hard for neural
networks to identify what is going on here.
So what kind of success rates can we expect?
70%?
Maybe 50%?
Nope.
2%.
Wow.
In a world where some of these learning-based image classifiers are better than us at some
datasets, they are vastly outclassed by us humans on these natural adversarial examples.
If you have a look at the paper, you will see that the currently known techniques to
improve the robustness of training show little to no improvement to this.
I cannot wait to see some followup papers on how to crack this nut.
We can learn so much from this paper, and will likely learn even more from these followup
works.
Make sure to subscribe and also hit the bell icon to never miss future episodes.
What a time to be alive!
This episode has been supported by Weights & Biases.
Weights & Biases provides tools to track your experiments in your deep learning projects.
It can save you a ton of time and money in these projects and is being used by OpenAI,
Toyota Research, Stanford and Berkeley.
In this post, they show you how to train a state of the art machine learning model with
over 99% accuracy on classifying squiggly handwritten numbers and how to use their tools
to get a crystal clear understanding of what your model exactly does and what part of the
letters it is looking at.
Make sure to visit them through wandb.com/papers or just click the link in the video description
and you can get a free demo today.
Our thanks to Weights & Biases for helping us make better videos for you.
Thanks for watching and for your generous support, and I'll see you next time!