hey Ingo do you know what time it is of course it's time for five minutes with
Ingo alright so today we will talk about something interesting in machine
learning which is called ensemble methods and the idea behind ensembles is
that you have a committee of experts every expert might be well know
something about this field but by combining all their opinions you get
actually a better well combined opinion like a compromise and those
compromises are typically very good in terms of for example of predictive power
so there are two basic techniques I mean there's multiple techniques for ensemble
methods but two very famous ones one is called bagging and the other one is
boosting we will discuss boosting a little later but bagging is is interesting
because we all can relate to to the idea behind bagging the idea is to ask
different people and we will do this actually in a couple of minutes ask
different people what do they think how many M&Ms are inside of this bowl so
they they will come up with a couple of estimates we asked over like ten people
and then at the end we take the average of all those estimates and you will see
something very interesting that the average actually is a pretty good
estimation so I can tell you now already inside this bowl there are 585 M&Ms so
let's see how well this committee of experts is actually or how close is this
committee of experts actually getting to this true actual number so let's get out
of here let's ask some people for their opinion and let's see how this
plays out let's go out here and ask this young man if he wants to give us some
estimate here
would you like to get a $5 Starbucks gift card all you need to do is tell me how
many sweets are in there just an estimate of course
738 738 say 900 a lot yeah a lot um 500 500 sounds like a perfect estimate 288 what do you think how
many are in there 450 1000 1000 um 120 900 900 325 okay so here we have our bowl with the 585 M&Ms and
we asked the bunch of people what they did they think how many M&Ms are in there
let's let's think about those numbers again in in well in order so we start
with the smallest number which was 120 was the lowest
estimate which obviously is pretty far off then we got 250, 288, 325, 450, 500
which is pretty close 738, 900, 900 again and then finally 1000 and again the 120
in the 1000 two ends of the spectrum are probably pretty far off but now the
interesting thing is if you what happens if you calculate the average the average
of all of those numbers is 547 think about this how close this average is to the
actual number 585 actually is it's only 38 M&Ms off and this is a concept which
we know as something we call the wisdom of crowds the wisdom of crowds is is a
good book I can recommend this and it's it's about like how the opinions of many
people can actually form something better than the opinions of individuals
so definitely worth reading not necessarily about machine learning but
it it follows the same concepts so now one important thing here is that even
the closest estimate which was 500 was kind of like 85 off so even the closest
was worse than the average which was only 38 off and that really is the power
of bagging so the other method I I mentioned in the car was called boosting so
the important thing for bagging is that all those opinions are actually
independent of each other but for boosting that is not the case so let's
for example say we have an expert for let's say green M&Ms
so this expert could have a look at those six green M&Ms here
how many are there no it's actually seven and this expert looks at them and
says like oh I'm an expert for green M&Ms let's say those are seven okay so
that's good and then we have another expert let's say an expert for orange
M&Ms and this expert here looks at this heap and now he can actually use what he
just heard before so this expert cannot say well if those have been seven then
I'm going with five here so those expert opinions now can build on top of each
other so they're no longer independent they actually reuse the expertise of the
others that is really how boosting or the idea of boosting works so it's not
independent they work on top of each other
so Ingo how do you implement boosting and bagging you might use it for machine
learning yeah well it's kind of straightforward so you have a bunch of
weak learners as we call them and so for example for for bagging you just take
ten different models and they don't need to be super good and then you basically
combine all those predictions so it's just the normal classification or
regression prediction let's say for classification then you just take the
majority of the selected class for regression you can use the simple
average of all the predictions so for boosting it's a bit different so here
you start with the first weak learner and then typically what happens let's say
for example for AdaBoost or others what typically happens is after you made a
prediction about some of the data space you weigh those examples down those data
points down so they no longer are really considered so the next expert then
focuses on the other data points which have not been considered before and
again they build then on top of each other
and I guess that's all for today on ensemble methods
thank you