Okay Michael. So, let's, let's write down a few equations.
Let's try to be a little bit more formal. A
little bit more mathematical about this idea. So, I'm going to
try to encapsulate, what we just talked about, to the
line of least commitment. By drawing another line. So, if
we think, of this top gray line, here. As sort
of the line, that gets as close to the plus
points as possible, without crossing over, to them, and mis-classifying them.
And we think of the bottom gray line, as
the one that gets as close as possible, to
the minus signs, without crossing over them and, and
giving mis-classification. And then the middle line, is sort
of in the happy medium. Okay? So, what you
really want from the lines, all the lines that
are possible, between ,these two boundary lines. If I
can use that language. Is that, somehow, this distance
here, is as big as possible. Can you see that?
>> Yeah, though it seems like the gray line, could
be pushed out a little more, right? because ,the minuses
don't bump into it. That's a good point Michael and
I'm going to fix that, by putting a minus sign here.
>> [LAUGH]
>> Okay [CROSSTALK]
>> I see, this is [CROSSTALK]
>> I did the best I could under the circumstances.
>> Data, revisionist history.
>> No, there's just an invisible point, and I just made it more visible.
For the sake of the reader. Okay, so, I've got these two
lines, these are sort of as far as I can go, without stating
to do mis-classification with my, my separating line, and the line in
the middle, we've already argued is the sort of the best one because
it provides the least commitment. So, that means, you want to have
a line, such that, it leaves as much space as possible, from the
boundaries. Alright, Michael. So let's see if we can figure out exactly,
what that line is like. So, the first thing, that I want to
do is, is introduce a little bit of notation.
Right? So we all remember what the equation, of
a line is. It's Y equals MX plus B,
that's just a general equation for a line. But,
here even though we're going to be drawing with lines,
we really want to deal with the general case,
where we're talking about hyperplanes. And generally ,when we
write about hyperplanes, we describe them as some output,
let's just call it y is equal to
w transpose plus b. here, because of what we're,
what we're trying to do with classification, the output
y is going to be some value that indicates,
whether, you're in the positive class or you're in
the negative class. W are the parameters ,or W,
represents the parameters for our plane. Along with b,
which is what moves it out of the origin.
Okay, are you with me.
>> I think so, but that's, so may be we should get rid of that top
,Y, because, that Y is different kind of
Y. Alright so the top Y is talking about
the Y dimension of the plane and in
the second equation, that Y is kind of folded
into the X. And we have a new
y, which is actually, the output of the classifier.
>> Right. I like it, so let's get rid
of that first y which is just an equation
for a line and let's ask what each of
these things are. So, let's just say that again
for clarity's sake. I think, you make a good point, Michael.
Y here, is going to be our classification label ,right, whenever we're
talking about using a linear separator, effectively, what we've been talking
about, which I realize now we never ever actually said explicitly, is
that you are taking some new point, projecting it onto the
line, and then looking at the value that comes out from projecting
it. And in this case, in particular, we want positive values
to mean yes, you are part of the class, and negative values
to mean that you aren't a part of the class. Okay?
>> Yeap.
>> This is our classification label y.
W represents again, the parameters of the plane.
Along with b, which is what moves it in and out of the origin. So, this is
now, effectively, what our linear classifiers actually look
like. Even in multiple dimensions, with hyperpoints. Okay? Cool.
So, let's take that, and and, and push it, to, to sort of the next level. Let,
let's figure out exactly, what we would expect the
output, of our hyperplane ,to be in this example,
that I, I've drawn on the screen here. So,
we know we want to find this orange line in
the middle ,which has the property, that, it is your
decision value, it tells you whether you are in the
positive class or negative class, on the one hand,
but also, that it has the property of being, as
far away, from the data as possible, while still being
consistent with it. So, if you're on the decision boundary,
for this particular line, which again,
is w transpose x plus b. What would be, the output
,of this classifier for any point that lies along the line?
>> so, right. If that's decision boundary, that's where it's kind of
not sure ,if it's positive or negative so that should be zero.
>> Right, so the equation of this line or this hyperplane, is w transpose x
plus b equals zero. For, some set of parameters w and b, we don't
yet know what they are. But, what we do know, that this is the definition
of a hyperplane. Where the equation for
a hyperplane, and since it's at the decision
boundary ,it should give me neither a positive or a negative output. Okay?
>> Yep.
>> Okay now, one question we can ask
ourselves then, if we look at these other
lines is well what's the equation for the
other grid lines? That are right at our positive
or negative examples. So to help you answer, that, I want to talk about what the
labels themselves ought to be. So ,just like
we did with boosting, let's say that our labels
are always going to be, from the set minus
one and plus one. We know ,that our labels
,are minus 1 and plus 1. And so we're
going to take advantage of that fact, by saying well.
The line that brushes up, against ,the positive example.
We want it to be the case ,since, we
know those labels are going to be plus 1.
That, the output ,of that particular line, that particular linear
separator, would be plus 1, on the very
first point, that it encounters. Does that make sense?
>> Yeah
>> Okay
>> That way the things, that are, that it's you
know that are kind of past the line are going to be above
1 and the things are before the line, in that
kind of demilitarized zone, are going to be between zero and 1.
>> Right, so in fact given what you
just said, what is the equation of that line?
>> Oh I see. So, it should, be the W transpose x plus b equals,
1 for the top gray line.
>> That's exactly right. And by a similar argument,
where would you say the, the line of the
hyperplane should be for the bottom gray line? Analogously,
it seems like that one should be minus one.
>> Right. So, we, we have the decision boundary,
we're looking at, and we know that the equation of
the line is w transposed x plus b is zero.
We know that if we slid that line, towards the
positive values, we would end up with w transposed
x plus b equals one. And if we slid it
towards the negative values, we'd end up with, equals
minus one, now we can ask ourselves, how this helps
us. And it helps us in a very a
simple way. We know that we want the boundary condition
line, the one that is actually our decision boundary, to
be as far as possible from both our positive and
negative examples, so that would mean then, and
I hope you buy this Michael, that the
distance ,between the two gray lines, which are
parallel, to that line, needs to also be maximum.
>> Yeah, that's exactly what we want.
>> Right, so we want this vector here, to have, the maximum link that we
can have. Okay, so, how are we going to
figure out how long, that particular line is?
So, here is a simple idea. Well, the
lines are parallel to one another. We can pick
points on that line ,to define, that particular distance
there. So, just because, it's really easy to do
the math, I'm going to chose a point here
and a point here. Those points have the property,
that ,if I draw the line between them, you
get a line ,that is, perpendicular, to the two
gray lines. Okay?And I don't know what those x values are, but
I do, I'm just going to call, them x1 and x2. That seem fair?
>> Hm, I guess that seems as good a name as any.
>> And that is going to define the two points that
I have, and the distance between them ,is in fact going to
be or the, the vector, that is defined by their difference
is in fact going to have the length that tells you how far
apart those two lines are. Which in turn
because of the way that we've constructed them Tells
you how far apart your boundary decision line is
from the data and we ant that to be
maximal ,because then we made the least commitment to
the data. So, let's write that down as algebra.
The equation for our positive line so to speak
,is ,therefore w transpose x1 plus b equals 1.
And all I've done there is substitute, some point, I don't have to know what is,
it's going to turn out, that gives me some point on that line, okay? It
gives me, puts me in some particular place, on that line. And similarly, I
can do the same thing, for my negative line. And get w transpose x2,
plus b equals minus 1. Now, we want the distance between,
these two, hyperplanes or these two lines, in this
example, to be maximal. And in order to figure
out what, that means, we need to know exactly,
what that line is. So, it's, it's the difference between
the two. So, we can just use our, our
favorite trick, when we're doing systems of linear equations.
And, just subtract the two lines. We basically have,
two equations and two unknowns. And, we simply subtract them,
from one another. So, that we can get
a single equation. That happens to represent, the
distance, between them. So, if I subtract the
two from one another, what do I get?
>> Quiz.
>> Oh, I like that, we get a quiz. That is the correct answer. Even
though it seems like a tightness match,
in fact quiz, is the correct answer [LAUGH]