Uh, excuse me, could somebody give me a cue there please?
("Connected" by Studio MC's plays) ...it's cold in here, is it cold in Germany?
Well it's actually quite nice now again it was supposed to rain a couple of days but
no, it's good!
OK, hey Ingo, sorry to interrupt but it's time for...
Five Minutes with Ingo!
Excellent, cool.
So today since I'm in Germany I will probably make it a little bit shorter topic and this
one is going to be a small trick every petitioner should really be aware of because it's very
helpful in practice to improve the prediction accuracy of your models.
So, before I'm getting there I need to show and explain something to you and I've drawn,
been drawing this here on this piece of paper.
So what you can see here is really, like the different areas for predictions.
So we have a decision boundary here in the middle and the further we are away from this
decision boundary the more secure, the more sure we are actually that the prediction is
correct.
So those plus cases here, we are pretty confident that those cases are really positive, so we
say we have 100% confidence and that is this quarter here.
But if you get closer to the decision boundary, the level of confidence goes actually down.
And in those cases here, maybe only have a confidence of 50% like in this quarter you'd
expect you'd only get 50% of all the cases right.
And the same is true on the negative front.
OK?
So that's this concept of confidence, and that's very useful in predictions in general.
Usually you are not just saying 'OK, like this is a positive case or this is a negative
case.
You're also interested in understanding how sure is the machine learning model actually,
that this case is really positive or really negative.
And the confidence can help you with this.
But there's another thing you can do, and I promise you a small trick to actually improve
your prediction accuracy... because let's think about this.
If you look at this picture again - so here you have our bar from before - and you look
at all the cases, then in all cases we are only correct 75%.
So that means that 25%, all those cases in the center here where we are less sure, we
are not correct.
So, the big question is: what can we do about this?
And that's now the small, neat little trick.
All right.
So we could only focus on those quarters which are further away from the decision boundary.
So we only take those two here and we just remove all the cases where the confidence
is not high enough.
And if you do this, we only cover 50% - those 25%, and those 25% - of the complete possible
dataset.
So, but in those 50% we do cover here, we are correct in 100% of the cases.
So instead of looking at all the datapoints, and then we end up being correct in only 75%
of the cases, we now look at 50% of the datapoints, but we are always correct, or well, as good
as we can be.
So why is this important?
Because often machine learning models are used to replace a human decision making.
That means that you can now automate those decisions for cases where you are 100% sure
and then the machine can make the decision and can act on that.
In all other cases when the confidence is not high enough, for those 50% when the machine
learning methods are saying like "Eh, maybe not, I'm not sure enough," those are the cases
which you can still manually work on.
So you reduce your manual work by 50% and that's often a big gain because those are
also typically the cases where it's human interaction not really necessary because they
are more clear cases.
And then you can give the less clear cases to a human being for the final decision, and,
yeah, and save some work with that.
And that's actually the project for today: Removing uncertain predictions.
And this has been your Fiiiiiiiive Minutes with Ingo!
OK, that's good enough for me!
So I'm looking like a pirate now?
Yes, exactly.
Yup!