Hey Charles, how's it goin'?
>> It's going pretty well Michael. How's it going with you?
>> Good, thank you.
>> Good, good, good. Guess what we're going to talk about today?
>> Well, reading off this screen, it looks
like maybe ensemble learning, and boosting, whatever that is.
>> Yes, that's exactly what we're
going to talk about. We're going to talk about
a class of algorithms called ensemble learners. And I think you will See
that they're related to some of the stuff that we've been doing already,
and in particular we're going to spend most of our time focusing on, boosting.
because boosting is my favorite of the
ensemble learning algorithms. So you ready for that?
>> Yeah! Let's do it.
>> Okay. So, I want to start this out by,
going through a little exercise with you. I want you
to think about a problem. Okay. And the particular
problem I want you to think about is, spam email.
>> Mm, I think about that a lot.
>> So, normally if we were thinking of this a classification
task, right, where we're going to take some email and we're going to decide
if it's spam or not. Given what we've talked about so
far we would be thinking about using a decision tree or
you know neural networks or kNN whatever that means
with email. We would be coming up with all of these sort
of complicated things. I want to propose an alternative which is
going to get us to ensemble learn. OK and here's the alternative.
I don't want you to try to think of some complicated
Rule that you might come up with that would capture spam email.
Instead, I want you to come up with some simple rules
that are indicative of spam email. Okay, so let me be
specific, Michael. We have this problem with spam email. That is,
you you're going to get some email message and you want some computer
program To figure out automatically for you if something is a
piece of spam or it isn't. And I want you to help
write a set of rules that'll help me to figure that
out. And I want you to think about simple rules, so can
you think of any simple rules that might indicate that something is spam?
>> Alright I can, yeah I can think of some simple rules. I don't think
they would be very good, but they might better than nothing. Like If for example
it mentions how manly I am, I, I would be, be willing to believe that
was a spam message. So like if the body of the message contains the word manly.
>> Okay, I like that. like that when body
contains manly.
I like that rule, because I often get non-spam messages talking about
manly. So I guess one man's spam is another man's normal email.
>> [LAUGH] I guess that's true.
>> Probably. Any other rules?
>> Sure if it, you know if it comes from my spouse it's probably not spam.
>> OK, so let's see, from spouse.
>> Her name's Lisa. Now we're going to call our spouse. So
let's say minus, I'm going to go to plus here, so we know
some rules are indicitive of being spam, and some
rules are indicitive of not being spam. Okay, anything else?
>> Possibly the length of the message. I guess. Like what?
>> I don't know. I don't know that this would be very
accurate, but I think some of this, some of the spam I
get sometimes is very, very short just like the, it's like the
URL. Like hey, check out this site, and then there's a URL.
>> Hm, I like that. So, we'll just say short.
Just contains URLs. Hm, I like those rules. Let's see if we can
think of anything else. Oh, how about this one. It's just an image.
>> Hm.
>> I get a lot of those where it's just an image.
>> I see, like in it's it's and if you
look at the picture it's all various pharmaceuticals from Canada.
>> Exactly. Here's one I get a lot.
>> Hm,
>> Lots of misspelled words that you end up reading as being
a real word.
>> Hm. But I don't know how I'd write that
as a rule. Or you could just list the words.
>> Like rules that, words that have already
been modified in that way. I guess so.
>> Yeah, kind of a, kind of a black list, a black list of words.
>> Okay so, words like, I would say manly, but you were saying prawn.
>> Or whatever that says. yeah, so they're and they're tons
of these. Right? I mean, another one that's, that's very popular
if you're old enough anyway is this one, remember this one?
>> Oh, sure that was sometimes a virus, right?
>> Yes. Our young, our younger viewers will not know this but this was one
of the first big spam messages that would
get out there. Make money fast. And there's
tons and tons of these. We could come up with a bunch of them. Now, here's
something they all kind of have in common,
Michael, and you've touched on this all ready.
All of them are sort of right. They're useful but no one of them is going to be
very good at telling us whether a message has
spam on its own. Right. So the word manly is
evidence but it's not enough to decide whether something
is spam or not. It's from your spouse, it's evidence
it's not spam, but sometimes you get messages from
your spouse that are in fact spam, because in fact,
she didn't actually send them. You know, and so
on and so forth. And sometimes she did email
from Princes in Nigeria. I didn't. And they're not
always spam. I, I actually do, but any case,
sometimes people are asking you for money, and maybe
that's message you want to ignore, but it isn't
necessarily spam. And some people are very interested in
getting like this and don't consider it spam, right?
>> So, so, okay, so I can see that these would
all maybe provide some evidence, but it seems really hard tp figure out the
right way of combining them all together to I don't know, make a decision.
>> Right, this is exactly right. And by the
way, if you think about something like decision tree, you
could. There's really a sort of similar problem going on
there. We can think of each of the nodes in
a decision tree as being a very simple rule and
the decision tree tells us how to combine them. Right?
So, we need to figure out how to do that
here and that is the fundamental notion of ensemble learning.
>> But wait, isn't, couldn't you also do something similar with
something like neural net. Right? Where each of these now becomes a
feature and we're just trying to learn weights for combining them
all together. So That would kind of satisfy what you were talking about.
>> True, I mean I think the the difference here in this case and
and I think you're absolutely right but
one difference here is that typically with the
new network we've already built the network
itself and the nodes and we're trying
to learn the weights whereas in something
like a decision tree you're building up rules
as you go along. And typically with ensemble learning you're building up a
bunch of rules and combining them together until you got something that's good
enough. But you're absolutely right. You
could think of neural networks as being
an ensemble of little parts. Sometimes
hard to understand, but an ensemble nonetheless.