I think by now we have a reasonable sense of what
linear dependence means.
So let's just do a slightly more formal definition of
linear dependence.
So we're going to say that a set of vectors-- Let me just
define my set of vectors.
Let me call my set s of vectors, v1, v2,
all the way to vn.
I'm going to say that they are linearly dependent.
If and only if.
So sometimes it's written if, if with a lot of f's in there.
So sometimes it's written if and only if.
Sometimes it's shown like an arrow in two directions.
If and only if I can satisfy this equation, I can find a
set of constants c1 times v1.
I can take a linear combination of my vectors all
the way to cn vn, that satisfy the equation that I can create
this into the 0 vector.
Sometimes it's just written as a bold 0, and sometimes you
could just write it -- I mean we don't know the
dimensionality of this vector.
It would be a bunch of 0's.
We don't know how many actual elements are in each of these
vectors, but you get the idea.
My set of vectors is linearly dependent-- remember I'm
saying dependent, not independent --is linearly
dependant, if and only if I can satisfy this equation for
some ci's where not all of them are equal to 0.
This is key, not all are 0.
Or you could say it the other way.
You could say at least one is non-zero.
So how does this gel with what we were talking about in the
previous video where I said look, a set is linearly
dependent if one of the of vectors can be represented by
the combination of the other vectors?
Let me write that down.
In the last few I said, look, one vector can be -- Let me
write it this way.
One vector being represented by the some of the other
vectors, I can just write it like this.
I can write it a little bit more math-y.
In the last video, I said that linear dependence means that--
let me just pick an arbitrary vector, v1.
Let's say that v1, you know this is arbitrary, v1 one
could be represented by some combination
of the other vectors.
Let me call them a1 times v -- let me be careful -- a2 times
v2 plus a3 times v3 plus all the way up to an times vn.
This is what we said in the previous video.
If this is linear dependence, any one of these guys can be
represented as some combination of the other ones.
So how does this imply that?
In order show this if and only if, I have to show that this
implies that and I have to show that that implies this.
So this is almost a trivially easy proof.
Because if I subtract v1 from both sides of this equation I
get 0 is equal to minus 1 v1 plus a2 v2 plus a3 v3 all the
way to an vn.
And clearly I've just said, well,
this is linearly dependent.
That means that I can represent this vector as a sum
of the other vectors, which means that minus 1 times v1
plus some combination of the other vectors is equal to 0,
which means that I've been able to satisfy this equation,
and at least one of my constants is non-zero.
So I've shown you that, if I can represent one of the
vectors by a sum of the other ones, then this condition is
definitely going to be true.
Now let me go the other way.
Let me show you if I have this situation that I can
definitely represent one of the vectors as
the sum of the others.
So let's say that this is true.
And one of these constants, remember it's not just this,
it's at least one, is non-zero.
So let me just assume, just for the sake of simplicity-- I
mean these are all arbitrary.
I'll do it in a new color.
Let me do it in the magenta.
Let me assume that c1 is not equal to 0.
If c1 is not equal to 0, then I can divide both sides of
this equation by c1.
And what do I get?
I get v1 plus c2 over c1 v2 plus all the way up to cn over
c1 is equal to 0.
And then I can multiply both sides of this, or I could add
negative v1 to both sides of this equation or subtract v1
from both sides.
And I get c2 over c1 v2 plus all the way up to cn over c1
vn-- there's a vn here --is equal to minus v1.
Now if I just multiply both sides of this by negative 1, I
get a minus, and all these become minuses and this
becomes a plus.
And I just showed you that if at least one of these
constants is non-zero, that I can represent my vector v1 as
some combination of the other vectors.
So we're able to go this way too.
If this condition is true, then I can represent one of
the vectors as a combination of the others.
If I can represent one of those vectors as a combination
of the others, then this condition is true.
Hopefully that kind of proves that these two definitions are
equivalent.
Maybe it's a little bit of overkill.
Let's apply that definition now, to actually test. You
might say, hey Sal, why'd you go through all of this effort?
I went through all of this effort because this is
actually a really useful way to test whether things are
linearly independent or dependent.
Let's try it out.
Let's use our newly found tool.
Let's say I have the set of vectors--
Let me do it up here.
I want to be efficient with my space usage.
So let's say I have the set of vectors 2,1 and 3,2.
And my question to you is, are these linearly independent or
are they linearly dependent?
In order for them to be linearly dependent, that means
that if some constant times 2,1 plus some other constant
times this second vector, 3,2 where this
should be equal to 0.
Where these both aren't necessarily 0.
Before I go up for this problem, let's remember what
we're going to find out.
If either of these are non-zero, if c1 or c2 are
non-zero, then this implies that we are dealing with a
dependent, linearly dependent set.
If c1 and c2 are both 0, if the only way to satisfy this
equation -- I mean you can always satisfy it by sitting
everything equal to 0.
But if the only way to satisfy it is by making both of these
guys 0, then we're dealing with a
linearly independent set.
Let's try to do some math.
And this'll just take us back to our Algebra 1 days.
In order for this to be true, that means 2 times c1 plus 3
times c2 is equal to -- when I say this is equal to 0, it's
really the 0 vector.
I can rewrite this as 0,0.
So 2 times c1 plus 3 times c2 would be
equal to that 0 there.
And then we'd have 1 times c1 plus 2 times c2 is
equal to that 0.
And now this is just a system, two equations, two unknowns.
A couple of things we could do.
Let's just multiply this top equation by 1/2.
If you multiply it by 1/2 you get c1 plus 3/2 plus 3/2 c2 is
equal to 0.
And then if we subtract the green equation from the red
equation this becomes 0.
2 minus 1 and 1/2-- 3/2 is 1 and 1/2 --of this is just 1/2
c2 is equal to 0.
And this is easy to solve.
c2 is equal to 0.
So what's c1?
Well, just substitute this back in. c2 is equal to 0.
So this is equal to 0.
So c1 plus 0 is equal to 0.
So c1 is also equal to 0.
We could have substituted it back into that
top equation as well.
So the only solution to this equation involves both c1 and
c2 being equal to 0.
So they both have to be 0.
So this is a linearly independent set of vectors.
Which means that neither of them are redundant
of the other one.
You can't represent one as a combination of the other.
And since we have two vectors here, and they're linearly
independent, we can actually know that this will span r2.
The span of my r vectors is equal to r2.
If one of these vectors was just some multiple of the
other, than the span would have been some line within r2,
not all of.
But now I can represent any vector in r2 as some
combination of those.
Let's do another example.
Let me scroll to the right, because sometimes this thing,
when I go too far down, I haven't figured out why, when
I go too far down it starts messing up.
So my next example is the set of vectors.
So I have the vector 2,1.
I have the vector 3,2.
And I have the vector 1,2.
And I want to know are these linearly dependent or linearly
independent.
So I go to through the same drill.
I use that little theorem that I proved at the beginning of
this video.
In order for them to be linearly dependent there must
be some set of weights that I can multiply these guys.
So c1 times this vector plus c2 times this vector plus c3
times that vector, that will equal the 0 vector.
And if one of these is non-zero then we're dealing
with a linearly dependent set of vectors.
And if all of them are 0, then it's independent.
Let's just do our linear algebra.
So this means that 2 times c1 plus 3 times c2 plus c3 is
equal to that 0 up there.
And then if we do the bottom rows-- Remember when you
multiply a scalar times a vector you multiply it by each
of these terms. So c1 times 1.
1c1 plus 2c2 plus 2c3 is equal to 0.
There's a couple of giveaways on this problem.
If you have three two-dimensional vectors, one
of them is going to be redundant.
Because, in the very best case, even if you assume that
that vector and that vector are linearly independent, then
these would span r2.
Which means that any point, any vector, in your
two-dimensional space can be represented by some
combination of those two.
In which case, this is going to be one of them because this
is just a vector in two-dimensional space.
So it would be linearly dependent.
And then, if you say, well, these aren't linearly
independent then, they're just multiples of each other.
In which case, this would definitely be a linearly
dependent set.
When you see three vectors that are each only vectors in
r2, that are each two-dimensional vectors, it's
a complete giveaway that this is linearly dependent.
But I'm going to show it to you using our dependent, using
our little theorem here.
So I'm going to show you that I can get non-zero c3's, c2's,
and c1's such that I can get a 0 here.
If all of these had to be 0-- I mean you can always set them
equal to 0.
But if they had to be equal to 0, then it would be linearly
independent.
Let me just show you.
I can just pick some random c3.
Let me pick c3 to be equal to negative 1.
So what would these two equations reduce to?
I mean you have just three unknowns and two equations, it
means you don't have enough constraints on your system.
So if I just set c3-- I just pick that out of a hat.
I could have picked c3 to be anything.
But if I set c3 to be equal to negative 1, what do these
equations become?
You get 2c1 plus 3c2 minus 1 is equal to 0.
And you get c1 plus 2c2 minus 2 is equal to 0.
Right?
2 times minus 1.
What can I do here?
If I multiply this second equation by 2, what do I get?
I get 2 plus 4c2 minus 4 is equal to 0.
And now let's subtract this equation from that equation.
So the c1's cancel out.
3c2 minus 4c2 is minus c2.
And then minus 1 minus minus 4, so that's minus 1 plus 4.
That's plus 3 is equal to 0.
And so we get our -- Let me make sure I got that right.
We have a minus 1 minus a minus 4.
So plus 4.
So we have a plus 3.
So that is a minus 2.
So minus c2 is equal to minus 3 or c2 is equal to 3.
And if c2 is equal to 3 and c3 is equal to minus 1-- Let's
just substitute here, so we get c1 plus 2 times c2, so
plus 6, plus 2 times c3.
So minus 2 is equal to 0.
c1 plus 4 is equal to 0.
c1 is equal to minus 4.
I'm giving you a combination of c's that
will give us a 0 vector.
If I multiply minus 4 times our first vector, 2,1, that's
c1, plus 3 times our second vector, 3,2 minus 1 times our
third vector, 1,2 this should be equal to 0.
Let's verify it just for fun.
Minus 4 times 2 is minus 8 plus 9 minus 1.
Yeah, that's minus 9 plus 9.
That's 0.
Minus 4 times plus 6 minus 2 that's also 0.
So we've just shown a linear combination of these vectors,
where actually none of the constants are 0.
But all we had to show was that at least one of the
constants had to be non-zero, and we actually showed all
three of them were.
But at least one of these had to be non-zero.
And I was able to satisfy this equation, I was able to make
them into the zero vector.
So this shows, this proves, that this is a linearly
dependent set of vectors.
Which means one of the vectors is redundant.
And you can never just say, oh, this is the redundant
vector, because I can represent this as combination
of those two.
You could just as easily pick this guy as the redundant
vector, and say, hey, I can represent this guy as the sum
of those two.
There's not one bad apple in the bunch.
Any of them can be represented by the combination of some
other, by all of the rest of them.
So hopefully you have a better intuition of linear dependence
and independence.
Maybe I'll continue.
I'll do a few more examples in the next video.