in this section we thought we would talk
about an advanced concept and that is
ways that machine learning has helped
improve traditional algorithms and in
particular this stack 20 cargo algorithm
is work that I have done in my research
so let's imagine that we want to find
something called the expected value of a function
that is if in many systems they
operate under different conditions and
we want to know how the system behaves
on average and one way to find out this
average thing is to do something that's
called a Monte Carlo procedure and if
you remember from way back during the go
section we talked about Monte Carlo tree
search and the Monte Carlo is the same idea here
So let's say we have a die and we want to figure out
what number we expect to get on average
weighting each number according to its
value so what we could do is we could
roll the dice a whole bunch of times
maybe the first time we get a two and
then we get a six and then we get
another two and another three and we
keep rolling it and maybe you find that
average value is three point four now
the true average value of a die is three
point five so we didn't get the exact
number but we did get pretty close now
this is also used for other systems so
for example let's say we have an
airplane and we want to know the average
drag on this airplane
now airplanes are really finely tuned to
behave well under a variety of
conditions so maybe when we flew our
airplane four times we found drag values
if say eight point one six eight point
one for another eight point one four and
then maybe an eight point one five
finding an average value of around eight
point one four now you could imagine
firstly that it's really expensive to
fly an airplane and so we want to find
this expected value this expected
performance using as few flights as
possible because each flight is really
expensive the other really key idea here
to note though is that with that die
example we found that with a couple of
rolls we had a pretty large error while
on the airplane example after a couple
of flights our error was only really
small and that's because the die numbers
vary all the way from 1 to 6 while with
this airplane performance example there
are much close to each other eight point
one six eight point one four so a key
concept
Monte Carlo is that the more your
simulations can vary the larger error
you can have in how bad you are
predicting the expected value so how
this works is on an aircraft we have a
lot of sensors that measure what the
conditions were when the flight was
taking place so we don't just know the
value of the drag for example maybe we
also know the Mach number I wish the
airplane was flying at that is how fast
the airplane was flying relative to the
speed of sound and so what we have is
this input-output relationship where the
input is how fast we are flying and the
output is the drag of the airplane and
again we want to know the average drag
over a bunch of flights so for example
we know how much fuel the airplane needs
and so on the right here we see a plot
where the x-axis has the Mach number
that we're flying at the y-axis is what
is the value of that drag and say the
blue line is the true value of drag
aught that Mach number now this isn't
actually how an airplane behaves so
don't go ahead and use this for your
designs but it does give us this
intuition where now what we can do is
take that input Mach number data and
learn what the drag is as a function of
that input data now maybe we come up
with this dotted green line that's in
the image now we see it's not an exact
match it's not a perfect fit for how
much drag there is but we see it's doing
a pretty good job even though we've only
had a small number of samples okay and
so now the key concept is what we can do
is we can take those two lines and
subtract them from one another and so
now we get this new what we will call a
regularized surface that is where we
have the true behavior what when we're
removing the effects that we can learn
really well and so what we have now is
this red line below that's the
difference between our the true estimate
and our machine learned value and now we
can Monte Carlo this object here and if
you remember Monte Carlo has a really
large error when the numbers changed by
a lot but by bringing in this machine
learned surface now the function is much
more smooth the wiggles aren't as big
and so our expected error is going to go
down by a lot just because we've reduced
what we would call the variance of the
function how many times
the function Wiggles around and so if
you'll pardon the one technical slide
the x-axis here shows how many samples
we have that is how many times did we
fly our airplane and the y-axis has what
about our estimate given that number of
samples on average so in order to
generate this plot I kind of ran this
idealized procedure a large number of
times to see how well worked now what we
see is that all of for all of these
lines the error goes down as we get more
samples right that makes sense the more
data points we have the better we're
going to do but what this also tells us
is the Green Line was if we just used
Monte Carlo we only took two average
values like we did with a dye the red
line shows what if we had just used the
machine learning data so we had taken
our samples fit machine learning and saw
the average performance of the machine
learning and then the blue line shows
this procedure that I told you where we
combine these two things together and
have this different Monte Carlo surface
and that's the blue line where we've
combined the two and we see that for all
numbers of samples so no matter how many
data points we had the stack Monte Carlo
estimate has a lower error so what we
saw here is that we took Monte Carlo a
technique that was invented during the
Manhattan Project in the 40s but by
using machine learning we were able to
augment this very simple idea and
decrease the error and the state for
example decreasing the number of
airplane flights we would need to get a
good enough estimate of the drag so go
back a couple of lectures back to
computers playing the game of Go we can
actually see that this kind of idea is
related to the alphago program and how
it was able to be so effective and so an
alphago they had this Monte Carlo tree
search where they played out random
games to measure a probability of
winning that's not the only thing they
did they augment adapt probability of
winning with what we called the value
function which estimated how good a
given position was and so by combining
these two ideas they regularize the
Monte Carlo and made the estimate of
winning much more accurate than
otherwise so this concludes our tutorial
thank you for joining our to me and I we
hope you've gained a good high level
overview of machine learning is and are
now well
to learn more one great way to learn
more is to look at the supplementary
material where we've included a lot more
information about the topics we've
discussed we hope that you've enjoyed
these lessons and we hope that you're
eager to learn more thank you