>> Earlier I mentioned sampling and I asked you whether that
sounded useful, and you said it was. So, let's do a
little exercise. Why? Why [LAUGH] is that a useful thing? Why
is it good idea to be able to sample from a distribution?
>> Well, because it's one of the two things that distributions are for.
>> What does that mean?
>> Well so why do you have a
distribution? A distribution is so that given some
value, you can, you can tell me what's
the probability of me seeing that value which
is kind of what it looks like when you have the probability function, but
also if you have a nice distribution
you can generate values according to that distribution.
>> Okay. That's a little bit circular in the sense that it didn't tell me why
it was useful to generate them other than it's one of the things you can do.
>> Well, you didn't ask me to actually make sense. But I mean, this is
the, the thing that you use distributions for.
Now why would you want to do that?
>> Yeah.
>> So,
if a distribution represents kind of a process, it would
be nice if I could duplicate that process, right? So, I
would have to be able to generate values in the
right way, consistent with the distribution in order to generate that
process. So it's like flipping a coin, or I want
to flip a coin and find out whether I'm going to
get heads or tails. It would be nice if I can
do that in a way that's consistent with whatever the underlying
bias of the coin is.
>> Okay, so yeah, if this distribution
represented something complex, we might, you know, for
whatever reason need to simulate that world
and, and act according to those probabilities. So,
yeah, that, that's a reasonable one. What else, what if, what if I showed you
this, if i took this distribution that
we used for the lightning and thunder example.
>> Mm-hm.
>> What if you wanted to get a handle on it? How can we
use sampling for the distribution to give
you some insight into how the storms work?
>> Okay
so let's see, I've, I've, I've got this representation
of the joint distribution, but it's just a representation of
the joint distribution. If I want to asked a
question like, well what's the chance that it's, oh let's
say, storming outside if I've heard thunder, I could
go through and, and, you know, back compute the reverse
of the conditional probability tables. And I could do things
like, or I could just generate a bunch of samples
where I had thunder and I can just see how
often the storm was also true. Does that make sense?
>> It does, though I'm not going to use
the words that you just used to write that down.
>> Okay.
>> I'm going to call that approximate inference. So the basic idea is that
you would like to do some inference, you'd like to figure out what might
be true of the world in different
situations. Instead of doing some complex probability
calculation, you're just going to imagine a
bunch of possible worlds and see how often
is it the case that whatever it is you
want to figure out is true. So yeah, that, that
turns out to be a really good way to
do it. In fact, sometimes I think that's a lot
of what people are doing when we're, when we're
making judgments in the world. We're just really, really good
at this kind of sampling from past realities that
are relevant, and we can make judgments based on that.
>> Hm. So, how would you do that?
>> How would I do what?
>> How would you do this approximate inference?
>> We're going to get to that but I wanted to.
>> Oh, okay, cool.
>> But
there, but there's one or two other
things about sampling that I wanted to mention.
>> Okay.
>> Another thing that I could imagine using this for
is this notion of visualization. Which may be, I mean this
in a, in a broader way than it sounds, not
necessarily to actually see what the distribution is like, but to
kind of get a feel for it. So, I bet if I was to run that if I was to draw
a bunch of samples from the lightening thundering set, you
would have a better feel for how likely different things are.
Just you as a person might get a sense of how these
things work. So, you can imagine in, in a medical domain a doctor
who's, who's thinking about prescri, prescribing a particular kind of drug for a
particular kind of person, if the
information about drug interactions and so forth
was, was represented as a big belief net, it might be hard to
look at it and know anything. But if it ge, if you use
that to generate a bunch of artificial patients you might start to get
to feel for oh, you know what, these kinds of people tend to
react badly in these kinds of circumstances.
>> That's still a kind of approximate inference, right?
>> That's right. So this is, this is a kind of an
in the machine sense, and this is kind of in the human sense.
>> Okay, I like that. So let's see, let's see
if I, if I understand this. So the, the nice thing
about the storm, the thunder, and the lightning example is that
it has pedagogical value. Because it's easy for a student to
look at that and go okay, I understand what's
going on here. One because there's only three nodes
and two arrows, and the other is because, we
think we understand how storms, thunder and lightning work. Right.
>> Yup.
>> Or most people do. So that makes a lot of sense. Of course
the downside of it is, we think we understand it. And so it's hard to
see why you would need to do samples, I mean, there's just a couple of
probability distributions and we kind of know
what it means. But in the real world,
there are perhaps hundreds and hundreds of variables
with complicated relationships and conditional independencies that, that aren't
necessary intuitive just by looking at the graph. And
so picking one conditional probability table and looking at
it isn't going to tell you much. But by
sampling I get real examples that are concrete that,
as a human being, I can understand without having
to, you know, really glock all the 25 different
conditional probability tables. Does that sound right? Is that. [CROSSTALK]
>> Yeah, yeah.
>> What you're trying to say?
>> That's exactly right. Thanks.
>> Okay.
>> I want to draw your attention to this, this
word here for a moment. This notion of approximate inference. Now
generally we don't like approximations when we can do things, things
exactly. So why are, why are we not doing things exactly?
>> because it's hard.
>> It's hard, that's exactly right. So or,
or, even if it weren't hard, it may,
it may be in some cases faster. So I would be,
I'm not going to do it now, but I'd be happy
if I guess if there's ground swell of support among the
students. To I can go through the argument as to why
this inference is hard. There's a nice little reduction to problems,
N, NP complete problems like satisfiability. But it turns out roughly
that if you could do inference exactly on any belief net
that you want, then you could solve very, very hard problems efficiently
using that idea. So it's, it's cute, but it's kind of takes us
a little bit off our path, so I'm not going to get into that.
>> Okay, so sampling is useful, Michael, which I always suspected in my
heart, and now we've got some good arguments for why it actually is.