Alright, so what we'd like to do is work up to an algorithm that can
actually do some of these inference steps instead
of having to think it through each time
de novo. So what I'm going to do is, let's hearken back to an example that we
looked at before which is about spam detection.
Do you, do you remember the spam example?
>> I do remember the spam example. That
was way back in the boosting lecture, right?
>> Yes, I think you did that one. I did, it was an excellent example.
>> There you go. So,
we didn't think about it in a Bayes net setting,
it was in a classification setting we were trying to
come up with the rule, but let's think of this
as a Bayes Net where there's a bunch of different variables
that can be true or false about any given email
message. It can either be spam or not. It can contain
the word Viagra or not. It can contain the word
prince or not. It maybe contains the word Udacity, or not.
>> Mm.
>> Right? And, so, just as we think about these as these random variables.
If we're trying to build a belief net or a Bayes net
with these variables. We have to say. kind of, what's dependent on what. In
terms of representing the probabilities. So how would you, how do you
think we should draw arrows to, to relate these to quantities to each other.
>> I think that the arrows should go down from spam to
the other features of spam mail and I'll tell you why. Because if,
I like this notion of generation that you talked about a
little bit earlier. It seems to me if you know. Spam
mail or not. It sort of generates certain words. And as
written as these are like words I mean I know the, the
spam example these are you know, kind of stand ins for
features. But they're sort of features of spam mail. Yeah I
think that's a really good way to think about it. So,
in some sense what we're saying if we draw the bayes net
in this way, then any given email
message has some probability of being spam. And
given that it's spam, it has some
probability of containing different sets of possible words.
>> Right.
>> So, I would say that, well what, so what do you, oh
let's see if we can actually fill in some of these values. So
given that we have a spam message, how likely do you think it
would be to contain a word like, well let's say the word viagra.
>> Fairly high.
>> It might
be 0.3, but a non-spam message might be, I don't know, like 0.001.
>> Right.
>> Something like that. So how about a word like prince?
>> Well I get a lot of email about Prince because I'm a Prince fan.
>> Yeah, I was thinking that. That's
why I thought it would an interesting example.
So, if in your spam messages, how likely is it for Prince to come up?
>> Fairly low.
>> Maybe like 0.2
because you're talking about the Nigerian princes
and whatnot. On the other hand among your
non spam messages how likely is it for prince to come up, do you think?
>> Well I get a lot of non spam, so,
its still relatively low, but not as low as .001.
>> Alright, so, let's say .1.
>> Okay.
>> That's a lot of prince spam.
>> You can never have enough prince spam.
>> Alright, so in the messages
that you have that are spam, how often does the word Udacity come up?
>> I guess, it's pretty low.
>> I don't think I've ever seen a spam
that mentions Udacity. Alright, what about your non-spam email?
>> Again, increasingly, it's getting higher and higher.
>> [LAUGH]
>> Almost as much as I get prince mail.
All right, so we'll call that .1 as well then.
>> Okay.
>> All right, so now we have, oh an,
an what's the probability of spam versus not spam?
>> [INAUDIBLE] Probability to have spam is pretty low,
I'm going to say, at this point, actually; it's not
that low. At this point, it's probably half my mail.
>> Wow. All right, I'm going to say .4
Alright, so this is now, Bayesian network structure
that actually is, it's not exactly generating spam,
but it is kind of capturing features of email
messages as they come in. So, we should
be able to answer questions like what's the
probability that a given message is spam, given
that the message has Viagra in it but not
prince or udacity. So, how would we work this out?
>> Well, Since it says Naive Bays I think I would use Bayes rule.
>> That would be naive of you. Now we have applied
Bayes rule, we have flipped things around, why is this giving
us an advantage? For this kind of network structure it actually
has a huge advantage because we can break this first quantity up.
>> Oh I do see that, so this is where those conditional independences
come into play If I'm reading this network right, each one of those attribute
values is conditionally independent of each other,
given that you know the value of SPAM.
>> Excellent.
>> So then that means that the first quantity there
is actually a product of each of those conditional probabilities.
>> Yeah, so this is a really convenient structure.
Because it really just decomposes into all these separate
helpful quantities. So in particular, we can actually derive
this by applying the chain rule. But what we end
up with is that this joint probability over these
three variables decomposes into a product of three independent joint
probabilities. The probability that's, Contains viagra given that it's
spam, which we have. That number is 0.3. That probability
that prince doesn't appear in it, given that it's
spam and that is that it doesn't contain prince given
that it is spam. So that should 0.8, cause 1 minus the
0.2. And that it's not udacity given that it's spam. Is
going to be 1 minus this 0.0001, should be 0.9999. All right.
So this is the case when things, when it is spam, and if it's not spam, we
can do this same thing and get a product,
and that we can normalize, to get what the,
the relative probabilities between it being spam and not spam. So then I'm a
big fan of normalization, but of course this makes me think about, since it's
sort of a classification problem, we only
really care about knowing which one's more
likely. We don't really care about the
probability, right? Do we have to normalize?
>> Yeah, yeah because we do care about the probability.
>> Oh we do?
>> Yeah because we're... I asked" What is the
probability of spam given these other quantities. Oh, I see.
>> But you're right. So the observation
that you're making is a really good one. Which is that we
can do probability calculations in this
setting, and that's actually going to give
us answers to classification problems. And we're going to connect this back to
machine learning. But but first let's write a general form of this formula.
>> Okay.
>> Because this this seems a little bit specific. Alright so
the general form for this, is that if we're trying to figure
out the probability of, of some kind of a a root node
like this, when you have all these little bristly things coming down.
You can think of it as a probability of a
value given a bunch of attributes. And that's going to be equal
to the product of the probability that each of those
attributes would be generated by that. Underlying this v. This, this
the label or the or the underlying class. Times the
prior probability that v and then we just normalize by all
the different possible values of, of v. This, this quantity across
all the possible types of v. So so this is one
way of actually getting a very general kind of inference done,
and there's, as you were pointing out, Charles, there's a. There's
a really nice reason to think about things in this form,
because it does let you do a kind of classification. So
essentially if you think of, of this top node as being
the class, this is what was playing the role of V
here, and these are all a bunch of attributes, then even
if, if we have a way of generating attribute values from classes.
What this let's us do is to go the other way.
That we observe the attribute values and we can infer the class.
>> Nice, so what's the equation for that?
>> Right, so the, the maximum oposterior
class if you're just trying to find whats
the most likely class given the, the data that you've seen. You can just take
an arg max over all the different possible values of that, that root node of
the prob, its probability times the product
of all the attribute values given that class.
So this would actually let us if you're, if you're been paying attention,
we could, in this particular case, compute map spam. Which is a palindrome.
>> Wow. That is spectacular.
>> You did not see that coming did you?
>> No I did not.