Professor Dave again, let’s get independent. With some basic definitions out of the way,
we now want to start thinking about ways we can express vector spaces in the simplest,
most efficient way. To do this, we need the fewest number of elements than can be used
to write any other elements within the vector space. To put it simply, we don’t need any
unnecessary information. Let’s learn how to do this now.
Let’s start by considering three vectors: a, b, and c. Let’s also say that c is equal
to (a plus 2b). In this case, for any linear combination of a, b, and c, such as 2a + b
- c, c can be replaced by (a plus 2b), making it a combination of only a and b. Because
c can be written in terms of a and b, it is unnecessary information, and the vectors a,
b, and c are said to be linearly dependent. That the vector c can be written in terms
of a and b, makes it “dependent” on them. On the other hand, if none of the vectors
can be written in terms of the others, they are linearly independent. The way this is
expressed is as follows. If you have vectors v1, v2, all the way to vn, in a vector space
V, then to be linearly independent, the equation c1v1 + c2v2 all the way to CNVN = 0 can only
be satisfied if all the scalars from c1 to cn equal zero. Let’s simply take this first
element over to the other side of the equation. By doing this we can see that the element
could be expressed in terms of all the others if any of the coefficients were nonzero. But
if they are all zero, then the left side simplifies to zero, and for the equation to be satisfied,
our last scalar must then also be zero. This is the requirement for linear independence.
So to see if a set of vectors or elements are linearly independent, we must simply see
if there is a set of nonzero scalars that exists that makes the linear combination of
the elements equal to zero. If those nonzero solutions exist, then the elements are linearly
dependent. If there is no solution aside from all zeros, the elements are linearly independent.
This ends up being the same as trying to solve a set of equations, which is a task we are
familiar with. Let’s start with a simple example, two vectors, (1, 1) and (1, -1).
As we just discussed, we’ll make a linear combination, multiplying these vectors by
scalars and setting the sum equal to zero. We will immediately see that we can rewrite
this as a set of two equations with two unknowns, which we could solve by substitution or elimination.
Opting for the latter, let’s just add these two equations together so that the c2 terms
cancel, and we will get two c1 equals zero, so c1 equals zero. Plugging this into either
equation we find that c2 must also equal zero. Our only solution for this system is all zeros,
and so our two vectors are linearly independent. This makes sense, because there’s nothing
we can multiply one vector by in order to get the other.
Solving a simple system like this is a piece of cake, but often we’ll be dealing with
much more complicated systems. However, we can still use the methods we’ve already
learned by treating the scalars as our unknown variables. If we solve the system and find
only zeros, the vectors are linearly independent, but if we find that any number of scalars
are nonzero, or that any scalar can be set to any nonzero value, the system will be linearly
dependent. When a variable can take on any value for the system, this is called a “free
variable”. The method we can use in such a situation involves setting up a matrix and
using row reduction to get it into row echelon form, so that the nonzero elements of the
matrix are only on the diagonal and upper right side, as we have learned previously.
To demonstrate, let’s consider the vectors (1, 1, 3, 3), (-1, 1, -1, -3), and (2, 5,
9, 6). We can take these vectors and multiply each by a scalar, and then set their sum to
zero. We can then turn this into a matrix equation and get to solving. In past examples
like this, we’ve included the right hand side as an additional column in our matrix,
making an augmented matrix. But since the right side is all zeros, we can actually just
ignore it in this case. Now recall that row echelon form has leading ones down the diagonal,
and all the elements below the diagonal are zero. We can use simple row operations to
achieve this. Our first element of the first row is already 1, which will make it easy
to get the elements below it equal to zero. Going row by row, first we will subtract the
first row from the second row to make the second row’s first element zero. Next we
will subtract 3 times the first row from the third to get a new third row, and we will
also subtract 3 times the first row from the fourth to get a new fourth row. Now the first
column looks great. Next, we want to make the second element of the second row equal
to one, while also making all elements below it zero. We can go ahead and subtract the
second row from the third to get a new third row, since all of the elements easily cancel
to give zeros. Now we can just divide the second row by two to make the leading element
equal to one. Looking at our solution, we have only two
rows with nonzero entries, which is one less than our number of unknown scalars. This implies
that we have a free variable that could be set to anything, meaning it’s not required
to be zero. Because of this, the set of vectors is linearly dependent. We can see this by
writing out our solution as a set of equations, like this. We end up with two equations with
three unknowns. This is not enough to constrain our unknowns, and we must have one free variable
which can relate one of our vectors to the others. So in this particular case, we conclude
linear dependence for these three vectors. If after our row operations we were left with
a surviving 1 in the third row, third column, when we wrote our set of equations, we would
have had to include a third equation that sets c3 equal to zero. Then, substituting
that into the second equation, c2 would have to be zero, and then plugging in zero for
both c2 and c3 in the first equation, we would find that c1 is equal to zero as well. All
three scalars would end up being zero, and this is how we would identify this as a linearly
independent system. We have one more method of determining linear
independence, and we can use this if we have a set of vectors that makes a square matrix.
This method involves finding the determinant, which is something we can only do with square
matrices. If the determinant ends up being 0, the vectors are linearly dependent, and
if the determinant is not zero, the vectors are linearly independent. Let’s go back
to our simple pair of vectors (1, 1) and (1, -1). We have already seen they are linearly
independent, so the determinant of the matrix made of these vectors should be nonzero. Let’s
set up the matrix and test it. To take the determinant we find one times negative one,
minus the quantity one times one, giving us a determinant of -2. Since this is not zero,
the rule tells us these vectors are linearly independent, which we previously verified.
Finally, given that vectors spaces can be made of more than just vectors, we can also
check for the linear independence of things other than vectors. For one last example,
let’s take a set of polynomials: x^2 + x - 2, x^2 - 3x + 5, and 2x^2 + 6x - 11. Just
like checking for independence of any elements, we want to multiply each by a scalar and set
the sum equal to zero. The most efficient way to proceed will involve grouping like
terms in powers of x, so we take the x squared terms and make this, then we do the same for
the x terms, and then for the constants. If this seems confusing, you can also just distribute
the scalars, group up like terms, and factor out the x terms if you prefer. Now, for this
equation to be equal to zero for all x, each coefficient must also be zero, so this first
coefficient must be set equal to zero, and then the second, and then the third. This
gives us three new equations involving our unknown scalars. While it did take a few extra
steps, we once again have a system of equations to solve just like we have been doing. In
this case, once we set up the coefficient matrix, we see that it is a square matrix,
so if we want to use our new trick, all we have to do is find out if the determinant
is zero, which would make the polynomials linearly dependent, or if the determinant
is nonzero, making the polynomials linearly independent. Going through and solving for
the determinant, a calculation which we will show all at once for our purposes here, we
find that it is equal to zero, and the polynomials are linearly dependent.
So vector spaces and related concepts certainly may seem abstract, and they are, but they
are useful because they allow us to unify things like functions, vectors, and matrices,
so that we can use the same set of tools to answer certain questions. Questions like:
Can this object be written in terms of other objects? This type of thinking, abstraction,
and unification, is prevalent in advanced mathematics, so we should probably just get
used to it now. With linear independence understood, we can move on and combine this with the concept
of span to form another key concept. But first, let’s check comprehension.