Well this...
This is productive work.
You know what Ingo, this would be the perfect time for Five Minutes with Ingo.
OK, sure!
So, we have a puzzle here let's think about this.
Graham, quick task for you.
OK.
I would like to ask you, describe 'puzzle', the concept of a puzzle to me, but without
using the word puzzle, can you do that?
Uhhhh, let's see.
Yeah, two hundred pieces.
Ooh, yeah.
OK, that's - yeah, fair enough.
That's a good idea.
So you say something with two hundred pieces, that's a puzzle.
OK, have a look at this here.
This here is a bowl with sugar bags, and those also have to be two hundred bags, so I say,
those are two hundred pieces, so therefore, that's a puzzle, right?
I'm gonna have to go ahead and sort of, disagree with you there.
Like you just said, something with two hundred pieces is a puzzle.
Right, but I know that's not a puzzle.
OK, well you have a point.
You as a human being can say that, but imagine how difficult it actually is for a machine
or for an algorithm to figure this one out, so lets have a look.
OK, so we have a couple of data points here, we have sen this line last episode already
that we can put data points into this two-dimensional space.
Yes.
So, actually I would like to explain those data points, finding a function which will
explain those data points for example like touching them, and so that you can use the
function in order to predict, 'what is the value, for example, somewhere here.'
Graham, another task for you, can you - well I'll open this up for you - can you actually
paint such a function, such a line explaining those data points?
You mean like connect them?
Yeah, for example yeah.
OK, let's see here.
That looks pretty nice, OK.
That looks great.
Uh yeah, well yeah yeah it's great.
So this is one way how you could do this, and I think that's probably something many
people would come up with.
But, so maybe we can come up with something else.
So, I for example could go with this function here, I could go up here, a little bit around,
and down again.
Oof.
Yep.
That's...
Yeah that's complex.
Yeah.
I know.
Non-linear, you might say.
Yeah, it's probably not really linear.
So, what are you thinking about this one?
I think it's messy.
Yeah, messy is a good description really.
You look at this and you think well, the green function here, that's good enough.
So why do you make things more complex?
Why are you making things more difficult?
Because, it's not really necessary, it's just like, you're adding complexity to this whole
function without any reason.
It's something people usually wouldn't do.
We usually try to make things as simple and easy as possible.
Alright, so then I have another idea here, so, thank you.
So, I could now say, maybe we can come up with something which is even simpler.
Could you come a little bit closer please?
Yes, thank you.
So let's go with this one here.
Look at that.
That one is even simpler.
It comes at a price though, because now we are no longer touching all the black dots
here, so we are not explaining every data point exactly.
But we are pretty close for all data points.
And the function is even easier because it's just linear.
That would be, by the way, the outcome of a linear regression model.
And we talked about regression like, a couple of episodes ago.
Which a little bit different from those classification tasks.
So anyway now we have this regression line and it's very very simple, but it comes at
a price that you're no longer correct for those data points, OK we got that.
I still think it's a great idea, why?
Because, in all those cases, like have a look, for example, here, at this area here.
If I were to ask 'Well here's a new point, and I would like to know what is the value
of this point?'
That would be great.
The red function would say, well, it's about this value, the green one pretty close as
well, maybe a good idea.
But look at the blue one.
That's totally off.
So that's not really great, and the reason for that is just because it's too complex
and you come up with solutions which are not very likely.
So, let's talk about the puzzle for a second again.
So you came up with the idea of saying that a puzzle is consisting of two hundred pieces.
Which is two complex.
That leads to wrong decisions for other objects.
So not a good idea.
And we call this over fitting.
That's a very important concept in machine learning.
What you have just did, you have been creating over fitting.
You're model was too complex too complicated for describing the puzzle, in the same sense
that this line here is too complex to describe those data points.
So it's very important to control over fitting, because otherwise your predictions are going
to be really really bad.
OK, that's it for me on over fitting today!
And this has been, your Five Minutes with Ingo!
So, if you can solve this challenge, you can win an amazing prize.
Data scientist number seven, show the prize.
Beautiful.
This puzzle here, one of a nice unicorn, autographed and signed by Data Scientist Number 7 and
me personally.
OK, you have to answer the following question: which of those two modelling techniques, linear
regression and K-Nearest Neighbors, is more vulnerable against outliers?
Think about this and tweet the result at @ingomierswa, thanks.