(The Alpha Go has been using Deep Learning to make this happen then?)
Yes, yes.
That's... I mean the company that did that, well, now is Google, right?
But it's Deep Mind company that was a startup in London and Google acquired them at some point,
before actually anyone knew what they were doing, right?
So they were smelling that it was something nice, right?
And, um, their, um -
Their best thing is that they're doing Deep Learning
with Reinforcement Learning. Right?
So Reinforcement Learning is the type
of way you try to teach,
and Deep Learning is the type of algorithm
that you use to try to learn that.
Right? So -
Reinforcement Learning is exactly this thing of,
I'm not gonna tell you exactly
what the output is.
I'm just gonna tell you, What is the task,
and I'm gonna tell you whether you're doing the task
well or bad. And it's gonna be your, uh,
your job, as an algorithm, to find the best way of doing this task.
The first idea that we have to understand is, what -
what we're trying to do with it, right, which is basically machine learning.
It's, ah -
It's trying to automatically
do some task that -
Well, we can do it by counting, usually, right?
We can do, basically, statistics.
But if we have really a lot of numbers,
we can end up
doing some heuristics,
what we can come up with.
Then we might want to use computers
to actually learn
from these statistics in an optimum manner -
Say, that the best way of
finding this information out of
these data that you have is following these rules, right?
So the computer does this for you.
This is the basic concept of machine learning.
One typical example is if you want to invest in, ah -
either an exchange market, in the stock market -
something like that; and, uh,
then, I guess the people that invest, they have
certain rules. Right? If you have
the last five weeks going up in value,
then, overall, you expect that
this week will go down, because it's being,
I don't know, like reaching a maximum;
or this kind of idea, right?
That, uh, you compute the
median of the last ten values, and
this gives you an idea of the *real* value,
and if it's above that, it will go down;
and so forth. Right? But this is just basically
heuristics that we *think* might work, right?
Then we can even, through experience,
think that some are better than others,
and so forth, right? But when we have, like,
50 values on the - of different stocks in the market,
and we have, you know, like
the currency exchange,
and we have many other factors, right?
How to - How many rules can we find there?
How many heuristics? Right? This is just, almost
infinite number. So, uh -
What you would like is,
ideally, a way of choosing the best.
And the computer has a capacity of checking them all.
Right? You can do that, pen and paper.
You'd spend, like, 5 lifetimes doing it.
And, maybe, you know - by the time
you finish, there's no stock market.
But the computer does it, everything for you,
very quickly. Right?
The only thing is, it will not do something better
than you can do, given enough time;
but it does it so much quicker
that you will never get there, right?
So now we begin to see cases in which
machine learning is actually doing
something better than humans, especially
with very specific tasks
and maybe
better than humans that are not experts
on that task. Right?
But, historically,
machine learning was always worse than
an expert human doing something. Right?
So, one of the cases is this,
like, now-famous AlphaGo. Right?
So, this is one of the few cases
in which machine learning has managed to beat -
to beat an expert. I mean, like
the world expert on one specific problem. Right?
And there's been this - all this
talking about why this is different from chess, right?
The main difference is that, for chess,
you have a limited number of options, right?
And, once you move one piece,
then you have, I don't know how many options, right -
but again, a limited number.
So, you can organize that in a tree,
and, more or less, check exhaustively.
You tell the program, This is what you want to do.
Check all the options, and just
do what is best according to
what I told you that is good. Right?
So, this is just a sheer competition of power
in order to check the options. Right?
This tells you, Our computers are really powerful.
Right? But it doesn't say anything about the algorithm itself.
Um, with Go, it's totally different
because there are so many options, that you cannot
really check them. Right?
A lot of people are repeating this fact, that there's
more Go positions than particles in the universe.
The people in Deep Mind, which is the people
that did this, have said that
they really don't know what kind of
tactic the computer is following. Right?
They didn't really hard-code any tactic.
So, what they did is, um,
they created this machine learning algorithm
that would play Go,
right, and would play terribly.
It's just, you know, some random rules.
They didn't really have a clue how to implement that,
so they started at some random
algorithm that would play Go terribly.
And the point is that they
pitted this, uh, this -
this machine, this algorithm,
against some other algorithm that would have
different parameters. Right? Would play Go differently.
But you still wouldn't hard-code anything.
You just would use some
parameters that would define how it would play,
but you really don't know what the meaning of
these parameters are. Right? And they would play against each other.
And at some point, after, I don't know, 100 matches,
one would win. Right?
And then you say, OK, this one is better.
And then, you keep these parameters, you change them somehow,
and you keep on doing that for a very, very long time.
It's a bit like evolution. There is a part of
computer science that is called evolutionary computing.
So I don't want to - this is -
this is slightly different thing. Right?
But the concept is, you have
a parameter space, right?
And this will tell you all the possible ways
to play Go, according to
your certain machine learning algorithm.
It will have certain possibilities
that are specific to that algorithm.
But, you have
a lot of parameters, and each
parameter can take a real value,
so, you have infinte possibilities.
And the question is how to find the best parameters
to play Go. Most machine learning algorithms
they have, you search for the optimum parameters
in some other way.
This - This was a specific of a
type of problems that are called
Reinforcement Learning, where you -
ah, you basically tell them
what you want to do, but, ah,
you are not sure what is the
correct way of doing it. Right? And,
that's why Deep Mind says,
We don't really know. Because they didn't
give the examples for the machine to imitate.
They said, OK, try to learn by yourself. Right?
This is what you want to do,
what we want you to do. Just learn how to do it.
Every time that the machine learning
decides a move, right?
So, this is what the algorithm is learning.
We have this configuration; what is my
next move? You are not specifying,
"The best move is *this*" because no one knows.
Right? I mean, the idea is that they beat
the world champion. So, you know, maybe
now, the algorithm was able
to figure out "Which next move is better?"
better than the world champion. Right?
So, there's no [golden standard to imitate here.] [unsure] Right?
That's the difference, in essence.
(I suppose I don't really understand the game Go,)
(so that doesn't help much.) Yes. (But I suppose, and the biggest thing)
(here is, um, feel free to kind of correct me,)
(but, in Chess, you can brute-force it.)
Yeah. Yeah. (And in Go, you can't.)
Yeah. (Would that be fair?)
That is - That is very fair. Yeah.
I mean, you still have to do it smartly,
to brute-force it for Chess, because you
have to find a way to tell the computer
whether a move is a smart
move or not, down the line. Right?
But you still would check all the consequences
of your move, right? But this is
kind of the standard. You put a bit
of domain knowledge, and you put brute force. Right?
The Go is a totally different game.
So, this is really fair assessment of it.
(So, basically, there were two sets of algorithms)
(working against each other; and every time)
(one did a bit better than the other, they thought,)
(Something about that is better.) Yes.
So, this, uh - There were two different
set of parameters, right,
of the same algorithm. Right? So the decisions
that it would take would be different.
The algorithm was the same. It was a neural network.
And, uh, basically,
what you tell is, the input is,
What is the configuration. Right?
And then you have a certain structure that will
you know, like, decide on the move. Right?
But this structure is, uh -
You need some parameters.
That is what, in the end, decides.
Right? What - So -
To see the difference, for example -
You can - You can see that you have a function
that will take the input and decide on the output.
This function, you can decide that it's a linear function.
Right? So, it's just a line, or a plane. Right?
This, uh -
The traditional, you know, like, regressing a line. Right?
And, um, this is,
when you modify the parameters of the line,
this is more or less the same.
You change the slope of the line. You change
the height of the line. Right?
And this gives you different lines, right?
So this is exactly the same, but
instead of having a line, it was, like, much more complex a function.
(So that would be in two dimensions.)
(But there are just so many more dimensions in it.)
Yeah, it was not only about the dimensions.
It's also about the way that
the parameters are structured. Right?
And this is why we use the word "deep" on it, right?
So, a line is what we call a
shallow structure
of the variables. Because you just have
some inputs, and you multiply
by some values, and you get the output. Right?
So, in a difference with deep learning,
or, well, deep algorithms in general,
is you have a certain input,
then you get some intermediate variables
by doing the same. It's like if you
fit a line, and then you fit another line,
and another line, and another line,
and then, when you input some values,
it will give you certain values. Right?
So you have, like, predictions. Right?
These predictions are not the ultimate thing that you
need; they are intermediate values
that, at the same time, input to another
layer of prediction. Right? And you
go doing like that. Right? So -
The input to one layer is the output of the previous layer. Right?
And this hierarchical structure is much more powerful
in the sense of what kind of functions,
what kind of - the flexibility of the functions. Right?
You can model much more complex things
with many less variables. Right?
And this is - This is a magnificent trade-off that gives it a lot of power.
You can go back and find historical examples;
endless historical examples,
of people claiming that something fantastic is right around the corner,
when in fact it isn't. [voices overlapping] We should get a shell. OK?
And we did. OK?
So that's a good start, right? We know our program works.