In the last video, we showed you the Cauchy-Schwarz
Inequality.
I think it's worth rewriting because this is something
that's going to show up a lot.
It's a very useful tool.
And that just told us if I have two vectors, x and y,
they're both members of Rn.
And they're both nonzero vectors.
And that was an assumption we had to make when we did the
proof, otherwise there was a potential of dividing by one
of their magnitudes.
So that would've been a big no-no.
But if we assume that they're both nonzero, then we can say
that the absolute value of their dot products is going to
be less than or equal to the products of
their individual lengths.
So that's the length of vector x and we defined that a couple
of videos ago.
And then this is the length of vector y.
And of course, this is just a regular number and then each
of these are just regular numbers.
They're not vectors once you take a length.
The length of a 50 dimensional vector could just
be the number 3.
It's just a scalar value.
So this is just scalar multiplication here.
And we also learned that the only time that this inequality
turns into an equality is the situation where x is equal to
some scalar multiple of y.
And so in some textbooks you'll say-- and this has to
be a nonzero scalar multiple.
But that's a bit obvious.
I told you that x and y are nonzero.
So if this was 0 then x would be 0.
And we already said that x is not 0.
But if you want to say there you could say that you know c
also is going to be nonzero.
But that essentially just falls out of
this information there.
But if this is the case and if and only if this is the case,
then we can say that the absolute value of the dot
product of the two vectors is equal to the
product of their lengths.
Now, this is all just a review of what I
did in the last video.
Now what else can we do that's useful with it?
So let's just play around a little bit.
I can't claim to be experimenting, I know where
this is going to go.
Let's see what happens if I were to take the
length of x plus y.
So I'm going to add the two vectors and then take the
length of that vector squared.
Well we know from a couple of videos ago that the length
squared can also be rewritten as the dot product of a vector
with itself.
This right here, x plus y, I know it
looks like two vectors.
But it's two vectors added to each other.
So it's really a vector. x plus y is a real vector.
I could graph x plus y.
So the length of x plus y squared, I can rewrite it as
the dot product of that vector with itself.
So x plus y dot x plus y.
And all of these are vectors.
These aren't just numbers.
And this is the dot product.
It's just not normal multiplication.
But we saw two videos ago that the dot product has the
distributive and the associative and the
commutative properties just like regular scalar
multiplication.
So you can kind of FOIL this out if that's how you remember
multiplying your binomials.
Or I will think of it more as just doing the distributive
property twice.
So this can be rewritten as x dot x.
Actually, let me write it as the distributive property
because that's sometimes not obvious to a lot of people.
So let me write this x as a yellow x and let me write this
whole term as x plus y.
So this right here can be rewritten as x dot-- so this x
dot this x plus y.
And then it would be plus this y dot-- I want
to just switch colors.
Plus this y dot x plus y.
It's good to see that when you're multiplying these,
you're just applying the distributive property.
All I did is I distributed this term along each of these
terms into sum right here.
So then I got this.
And then I can distribute each of these into the sum.
So then this becomes-- I'll be careful with the colors-- x
dot x plus x dot y.
Maybe this was a little bit overkill, but I think it's
good to see that this isn't just some magic here.
And we're just using the exact properties that we proved with
the dot product.
So that's that right there.
And then it's plus y dot x.
Plus this yellow y dot the yellow x.
Sorry, dot the blue y.
So the magnitude or the length of our vector x plus y squared
can be rewritten like this.
And I'll just switch back to one color.
So this equals that and all of that-- what does this equal?
This is equal to x dot x.
What's x dot x?
x dot x is just the magnitude.
So let me write this is just equal to the magnitude
of our vector x.
I should stop using the word magnitude.
The length of our vector x squared.
And then I have two terms here.
I have an x dot y and a y dot x.
We know that x dot y and y dot x are really the same thing.
We showed that order doesn't matter when you take the dot
product, just like it doesn't matter with regular
multiplication.
So these are really the same terms. So we could write plus
2 times x dot y.
And then finally, we have that last term sitting here.
We have this y dot y.
y dot y is the same thing as the length of
our vector y squared.
Now, let's see if we can break out our Cauchy-Schwarz
Inequality.
Or maybe Schwarz, I don't know if I'm pronouncing it right.
But x dot y.
t We have the absolute value of x dot y here.
But we know that just x dot y is going to be-- it has to be
less than or equal to the absolute value of x dot y.
Why is that?
Well this could be negative.
I could show you examples of dot
products that are negative.
In fact, if x has all positive terms and y has all negative
terms, you're going to have a negative dot product.
So this could be negative or it could be positive.
If it's positive the absolute value-- their
equal to each other.
If this is negative, than this absolute value is definitely
going to be greater than it.
We can add to the Cauchy-Schwarz Inequality and
this is a bit obvious.
We could say look, we could add a little x dot y is less
than or equal to the absolute value of x dot y.
Which is less than or equal to the length of x times the
length of y.
So x dot y is definite-- this, the dot product of x with y is
definitely less than it's absolute value of that.
Which is definitely less than the lengths of those two
multiplied.
So if I rewrite this, this statement right here is
definitely less than or equal to this exact statement.
But if I replace these with the lengths of the vectors.
So that is definitely less than or equal to-- I'm just
rewriting this x squared and I'll write the plus 2 there.
Plus 2.
But I want to make it very clear what I'm replacing here.
And then I have the plus length of my
vector y there squared.
Now this I'm saying, this is definitely less than the
absolute value of x dot y.
Which is definitely less, by the Cauchy-Schwarz Inequality,
definitely less than the product of the two lengths.
So I'm just replacing this with the product
of their two lengths.
So I'm going to put the length of x times the length of y.
And since this is the same as that, this
is the same as that.
But this is definitely less than this.
This whole term has to be less than this whole term.
Now let me just remind you what we were doing.
I said that this thing that I wrote over here, this is the
same as that.
So this thing up here, which is the same as that, which is
less than that also.
So we can write that the magnitude of x plus y squared
and not the magnitude, the length of the vector x plus y
squared is less than this whole thing
that I wrote out here.
Or less than or equal to.
Now, what is this thing?
Remember, I mean this might look all fancy with my little
double lines around everything.
But these are just numbers.
This length of x squared, this is just a number.
Each of these are numbers and I can just say hey, look, this
looks like a perfect square to me.
This term on the right-hand side is the exact same thing
as the length of x plus the length of y.
Everything squared.
If you just squared this out you'll get x squared plus 2
times the length of x times the
length of y plus y squared.
So our length of the vector x plus y squared is less than or
equal to this quantity over here.
And if we just take the square root of both sides of this,
you get the length of our vector x plus y is less than
or equal to the length of the vector x by itself plus the
length of the vector y.
And we call this the triangle inequality, which you might
have remembered from geometry.
Now why is it called the triangle inequality?
Well you could imagine each of these to be
separate side of a triangle.
In fact, let's draw it.
We can draw this in R2.
Let me turn my graph paper on.
Let me see where the graphs show up.
If I turn my graph paper on right there, maybe
I'll draw it here.
So let's draw my vector x.
So let's say that my vector x look something like this.
Let's say that's my vector x.
It's a vector 2, 4.
So that's my vector x.
And let's say my vector y-- well I'm just going to do it
head to tail because I'm going to add the two.
So my vector y-- I'm going to do it in nonstandard position.
Let's say it's look something like-- let's say my vector y
looks something like this.
Draw it properly.
That's my vector y.
What does x plus y look like?
And remember, I can't necessarily draw any two
vectors on a two-dimensional space like this.
I'm just assuming that these are in R2.
But this is just to give you the idea.
So then this is their sum, right?
You took from the tail of x to the head of y.
So this vector right here is the vector x plus y.
And that's why it's called the triangle inequality.
It's just saying that look, this thing is always going to
be less than or equal to-- or the length of this thing is
always going to be less than or equal to the length of this
thing plus the length of this thing.
And that's kind of obvious when you just learn
two-dimensional geometry.
That look, this is a much more efficient way of getting from
this point to this point than going out here and
then going out here.
And then, what's the case in which this length is equal to
these lengths?
Well if you keep flattening this triangle out and you go
to the extreme case where maybe the vector
x looks like this.
And if the vector y is just kind of going in the exact
same-- vector y is going in the exact same direction,
maybe it's going a little bit further.
This is vector x, this is vector y.
Now x plus y will just be this whole vector.
Now that whole thing is x plus y.
And this is the case now where you actually-- where the
triangle inequality turns into an equality.
That's why that little equal sign is there.
The extreme case where
essentially, x and y are collinear.
And why does that work out?
We can even go back to our math and understand that.
So let me turn my graph off.
We can go back to our math here.
If I go back to this point, remember, right here I made
the statement, look, this thing is definitely less than
this thing over here.
But what if I made an assumption?
What if I said that x is equal to some scalar times y?
And actually, I have to be careful because just some
scalar times y-- remember, our Cauchy-Schwarz Inequality said
that look, the inequality turns into an equality if x is
some nonzero scalar of y.
And then we can apply this.
We can say that the absolute value of x dot y is the same
as this over here.
But I don't have the absolute value of x dot y here.
I don't know that this is positive.
I can say definitively that this is a positive quantity
because I took the absolute value of it.
No absolute value here.
So the only way that I can assure that this is a positive
quantity, that this is the same thing as the absolute
value of x dot y is to enforce-- if I'm kind of going
to go down this road, is to enforce that this term right
here, that c be positive.
Because if c is positive, then x dot y, if x dot y then that
would be the same thing as cy dot y.
Which we know is just the same thing as c times the magnitude
of y squared.
And the only way that I can ensure that this expression
right here is equal to the absolute value of x dot y, the
only way I can assure this is that c is positive.
If c is negative, then this is going to be a negative number
while this is a positive.
So if I assume that this is positive, then I can say that
x dot y is equal to the absolute value of x dot y.
And since it's a scalar multiple, then I could say
that that term is equal to, not just less than or equal
to, the magnitude of x's and y's.
So hopefully I'm not confusing you.
So all I'm saying is, if I can assume that x is some positive
scalar multiple of y, that this wouldn't
be a less than sign.
Then I could say that x dot y is the same thing as the
absolute value of x dot y since this is positive.
And if it's the same thing as the absolute value of x dot y
and it's some scalar multiple of each other, than we could
go down this other route.
We could say that this thing here-- I don't
want to get too messy.
We could say that this is equal to that.
If this is equal to that, then this would have become an
equal sign, not a less than or equal to sign.
And then we would have had the limiting case-- I don't want
to call it the limiting case.
But we could say that x plus y-- we would've done the same
work, but we would've had an equal sign the whole way.
Would equal the length of x.
The length of x plus y would equal the length of x plus the
length of y in the situation where x is equal to some
positive scalar times y.
So c is greater than 0.
These two imply each other.
And we saw that geometrically.
I lost my axes here, but we see that the only time that
the length of x plus y is equal to the length of x plus
the length of y is when they're collinear.
Over here this plus this is clearly-- you can just
visually look at it-- longer than this right there.
So you might be saying Sal, once again, this linear
algebra's a little bit silly.
We learned the triangle inequality in
eighth or ninth grade.
Why did you go through all of this pain to redefine it?
And this is the interesting thing.
What I just drew here and this is what you learned in ninth
grade geometry.
This is just in R2.
This is just your Cartesian coordinates, or I don't want
to use the word dimension too much because we're going to
define that formally.
But this is kind of your two-dimensional space
that's going on.
What's interesting or what's useful about linear algebra
is, we've just defined the triangle inequality for
arbitrarily large vectors, or vectors that have an
arbitrarily high number of components.
Each of these, these don't have to be in R2.
This is true if we're in R100 where every vector has a
hundred components to it.
We've just defined some notion of the triangle inequality.
We've abstracted well beyond just our two-dimensional
Cartesian coordinates.
Well beyond even our three dimensions to essentially, n
dimensional space.
And I haven't defined dimensions yet, but I think
you're starting to appreciate what they are.
But anyway, hopefully you found that useful.
We can now take this result.
And actually, that result with this result and define what
the notion of an angle between two vectors are.
Once again, you know, on some levels you're like well, why
do we have to define an angle?
Isn't an angle just-- isn't that just an angle?
Well yeah, we know what an angle is in two dimensions,
but what does an angle mean when you abstract things to n
dimensions?
Or when you're in Rn.
So that's what we'll talk about in the next video.