Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
This is going to be a weird, non-traditional episode.
Not the usual Two Minute Papers.
Hope you’ll enjoy it and if you finished the video, please let me know in the comments
what you think about it.
So let's start.
Life lessons I learned from ai research.
Number one, you need an objective.
Before we start training a neural network to perform, for instance, image classification,
we need a bunch of training data.
This we can feed to this neural network, telling that this image depicts a cat, and this one
is not a cat, but an ostrich.
We also need to specify a loss function.
This is super important, because this loss function is used to make sure that the neural
network trains itself in a way that its predictions will be similar to the training data it is
being fed.
It is also referred to as an objective or objective function to indicate that we know
precisely what we are looking for and that’s what the neural network should do.
This is a way to measure how the neural network is progressing, and without this, it is useless.
Similarly, in an other learning problem, we specify an objective for this ant, which,
in this case, is to be able to traverse as far from a starting as possible, and it reconfigures
its body type and movement to be able to score high on our objective.
And, this leads us to the second lesson: a change in the objective changes the strategy
required to achieve it.
Look here - in a different problem definition, we can specify a different objective, for
instance, a different terrain, and you see that if we wish to succeed here, we need a
vastly different body type.
Form follows function.
And in this other case, the objective is to be able to traverse efficiently, but with
minimal material use for the legs.
The solution, again, changes accordingly to the objective.
New objectives require new strategies.
Number three: If the objective was wrong, do not worry, and aim again!
Have a look at AlphaGo.
This is DeepMind’s algorithm that was able to defeat some of the best players in the
world in the game of Go.
This was a highly non-trivial achievement as the space of possible moves is so stupendously
large that it is impossible to evaluate every move.
Instead, it tries to aggressively focus on a smaller number of possible moves, and tries
to simulate the result of these moves.
If the move leads to an improvement in our position, it is a good one, if not, it should
be avoided.
Sounds simple, right?
Well, we have an objective, that’s great, but, initially, it has a really bad predictor,
which means that it is really bad at judging which move is good and which one isn’t.
However, over time, it refines its predictor and these estimations improve further and
further.
In the end, by only taking a brief look at the state of the game, it can predict with
a high confidence whether it is going to win or not.
Initially, we have an objective, but how do we know whether it is a good one?
Well, we try to get there, and then, evaluate our position.
We may find that we got nowhere with this.
What most people do is abort the program.
Quit the game.
Give up.
It’s over.
No, don’t despair, it’s not over!
This is the early stage of teaching an AI, and this is the time where we can improve
our predictor, and pick our next objective more wisely.
Over time, you’ll find the ideas that don’t work, and not only that, you’ll find out
why they don’t work.
Do not worry and aim again.
And this leads us to lesson number four: zoom out and evaluate!
This is exactly what DeepMind’s amazing Deep Q-Learning algorithm does that took the
world by storm as it was able to play Atari Breakout on a superhuman level just by looking
at the pixels of the game.
This algorithm ran in two phases, where phase one is collecting experiences, and phase two
was called “experience replay”.
This is where the AI stops and reflects upon these experiences.
Zooming out and evaluating is immensely important, because after all, this is where the real
learning happens.
So every now and then, zoom out, and evaluate.
And while we are here, I can simply not resist adding two more lessons I learned from other
scientific disciplines.
So, lesson number five: if you find something that works, hold on to it!
This is exactly what Metropolis Light Transport does, which is a light simulation algorithm
that is able to create beautiful images even for extremely difficult problems where it
is challenging to find where the light is.
However, when it finally finds something, it makes sure not to forget about it and explore
nearby light paths.
It works like a charm for difficult light transport situations and can create absolutely
beautiful images for even the hardest virtual scenes.
Seek the light and hold on to it.
And whenever you feel that you are still not making progress, think about the following.
Lesson number six: as long as you keep moving, you’ll keep progressing.
Take a look at this random walk.
A random walk is a succession of steps in completely random directions.
This walk is completely lack of direction, just as a drunkard that tries to find home.
However, get this - a mathematical theorem says that after N steps, the expected distance
from where we’ve started is proportional to the square root of N.
This is huge!
What this means, is that for instance, it we took four completely random steps, we are
expected to be 2 units of distance away from where we’ve started.
That’s progress!
If we take a hundred steps, even then, we can expect to be around 10 units of distance
away from the starting point.
This concept works, even if our predictor is completely haywire and we choose our objectives
like a drunkard.
Now I think that’s a lesson worth sharing.
To recap: you need an objective!
It can be anything, so long as you keep moving, you’ll progress.
If you have achieved it and it ended up not being what you were looking for, don’t stop.
Zoom out and reflect.
This will help you to improve your predictor and you will be able to recalibrate and aim
again at something more meaningful.
Now, aim and find a new objective!
When you have a new objective, your strategy needs to change to be able to achieve it.
Finally, if you find something desirable, hold on to it, and explore more in this direction.
Seek the light.
Of course, you don’t have to live your life this way, but I think these are interesting,
mathematically motivated lessons that are worth showing to you.
After all, this series is not only meant to inform, but to inspire you to get out there
and create.
It always feels absolutely amazing getting these kind messages from you Fellow Scholars,
some of you said that the series has changed your life in a positive way.
I am really out of words and am honored to be able to make these videos for you Fellow
Scholars.
Let me know in the comments whether you enjoyed this episode and please, keep the kind messages
coming, they really make my day.
Thanks for watching and for your generous support, and I'll see you next time!