So it is important to get an example I, I think
that the, the best way to kind of see an example
here is to draw a little picture for you that tells
you what the sort of underlying assumption behind ICA is. So here's
the kind of, or at least the way I see what
the fundamental assumption of ICA is. And that is this. You're, you're
assuming that the, there's a bunch of. Things in the world.
And these are going to be hidden variables. And they've got some properties
that, that are associated with them. Their random variables. They
are mutually independent of one another. That is if I
know the value of one it doesn't tell me anything
about the value of the other. But we don't know what
they are. And what we get to see are observables. This observable are given rise
to by the value of the hidden variables and they somehow combine in
a sort of linear fashion and our job, the job of any learner
in this case, any unsupervised learner in
this case is given your observables try to
find the hidden variables. Under the assumption that
these hidden variables are in fact independent of
one another. So, what's a concrete example of
that? I'm going to give you a concrete example
of that then I'm going to give you a demo that we found on the web. Okay?
>> Okay.
>> So, one problem where we know this is true is
something called the blind source separation problem. So what' s the Blind
Source Separation problem? It also has another name which is
the Cocktail Party Problem do you remember the Cocktail Party Problem?
>> I think so, yeah.
>> So, describe it to me.
>> So, not that I go to a lot of cocktail parties, but if you're
in a, in a large group of people like in a cafeteria or something like that.
>> Mhm.
>> And you're trying to listen to a conversation. It can
be very challenging because theres
all these different conversations happening simultaneously.
>> Mm-hm.
>> And we need
to be able to pull out that one source
that, that we're, that we're caring about, so that
we can listen to it while kind of separating
it out from what all the other noise sources are.
>> So what you just described to me is
that somehow you have a bunch of people, so
there's Charles, there's Michael and there's a push car.
And there all talking at once. And let's a imagine
in place of ears we have microphones. And these
microphones are actually recording everything that well everything that
they hear. So this means that this first microphone
is going to hear me. The second microphone is also going to
hear me and so does the third microphone... However,
they're going to each hear me at a slightly different
volume at a slightly different delay. Okay? Because they're
placed sort of randomly around the room. You with me?
>> I think so, yeah.
>> Okay,
surely, Michael, who just keeps talking and talking and
talking mostly about puns, is also going to be picked up
by each of the three microphones, and Pushgar, mainly complaining
that we aren't doing enough quizzes, is also going to
be heard by the three microphones. In this case if
we look at our other examples, the people are the
hidden variables. That is, they're the things that are causing
events to happen in the world. But what we have
are observable's. Are the microphones, and what it is that
they're recording. Now, if we're in a small enough room, it
turns out that physics tell us that, even though our voices
are all nonlinear and sound doesn't travel in a very linear
fashion the way that we would like, it turns out
that in a small enough room with all of the reflections
and everything, each of these microphones are well modeled as receiving
unknown linear comment, combination of all of the sources. So this
means that each one of these microphones actually contains a, a
different linear combination of each of our voices. Hey, you with me?
>> Yeah, it's interesting.
>> Right. And so, really, if I were to give you these three recordings
and I wanted to figure out what say, Michael Lipman was saying, all that
information is here, but I can't extract
Michael Lipman from microphone one, or microphone
two, or microphone three. But given all
of these microphones, I can in fact recover,
just Michael alone or just Charles alone or just
Pushcar alone, because ICA exactly tries to find this
notion of mutual independence without losing any information form
the output. And this model here, of three people
talking, mostly independently, and being recorded by linear combination
by different microphones is exactly the model that ICA
was designed to represent. So, just to sort of
drive this home, let me give you an actual example
that we found on the web that they do exactly this problem. Okay?
>> Sure.