This video is sponsored by Brilliant.org and it’s the second video in a mini series about
Linear Algebra. Click here and watch the first one on vectors and bases for vector spaces.
This video is about Matrices. When I first met a matrix it was in a textbook question.
It asked me to put a bunch of numbers to do with a bakery’s sales in an array. I was
very confused. It took me a long long time to really understand that matrices have really
nothing to do with bakeries and that just the same way that writing a vector as a column
of numbers is just a shorthand, writing a matrix as an array of numbers is a shorthand
too.
This is what Matrices really are: they are linear transformations.
Ok so let me explain. Remember vectors from last time? Well imagine you have a some vector
and want to transform it into some other vector instead. In fact say you have some whole vector
space, let’s say all the vectors in 2D space, and you want to transform them into some vectors
in another vector space or the same space, for example vectors in 3D space. Then a transformation
is just something that eats a vector from your initial space and spits out a vector
in your new space.
A Matrix is a transform. But a very specific type called a linear transform, which is something
that respects linear combinations. Let’s say that I have two vectors. I’m going to
apply this matrix to them each and that will transform them in some way. Then, I can make
a linear combination of the result. But what if I decided to do this linear combination
first, and then put the resulting vector into the matrix.
A linear transformation is one where these to processes result in the same final vector.
In other words, you can first put your two vectors into the matrix, and then take a particular
linear combination of them or you can take the linear combination of them first, then
apply the matrix. Either way you will get the same result.
To see if you’ve understood, I want you to figure out if the following transformations
are linear transformations or not. The first is a transformation that takes a
2d vector and then just multiplies its length by 3.
The second is like this. First draw a line through the origin. Then take any vector in
2d, say this one, and draw a perpendicular from that line to the tip of the vector. The
new vector is on the line.
So are these two linear transformations? Think about the answer and then put it in the poll
in the corner. Remember that committing to an answer like this helps you learn.
Ok, so the answer might surprise you but the answer is that they’re both linear. To see
this for the first one, take any two vectors and apply the transformation to them then
make a linear combination of them. This is clearly the same as if you made the linear
combination and then multiplied the result by 3. The second one is a bit tricker, but
the easiest way to see it is, first draw two vectors. Then notice that these vectors are
equal to these two perpendicular vectors added together.
When you apply the transform to them, what it does is it cuts off this bit. Then if you
add them you get this. But if you’d instead added the original vectors first, it would
be like adding all these 4 bits first, which you can group up like this. Now if we apply
the transform it cuts off this bit, and see, we have the same thing in both cases. Kind
of neat right?
But here’s a really nice thing about linear transforms. Imagine you have a linear transform
from some initial space, to some other space and you have an arbitrary basis for your initial
space Now let’s suppose that we know exactly what
the linear transform does to the basis vectors. This is in fact enough to figure out what
it does to the entire space. To see why, say you have some other vector in this initial
space. You can write it as a linear combo of basis vectors- in 2d remember you do this
by using
a grid. Ok, once you’ve finally got that linear combination if you apply the linear
transformation to it, you can see by the linearity that you only need to know what happens to
the basis to compute this thing.
So now we get to writing a matrix as an array. What you’ll see is, this first column represents
what happens to your first basis vector, the second what happens to your second etc. Remember
how to write vectors as columns in some basis? It’s a shorthand for writing out this linear
combination of basis vectors. Well then clearly the basis vectors are written like this.
Say I apply my linear transform to the first basis element and I get this column vector,
and I do this for each basis vector. As we said before this is everything I need to know
about this linear transform, so I want to store this information compactly, so I store
it in a matrix like this. And that is really all a matrix is. It’s shorthand, where,
just like in the case of vectors written as columns, the basis is hidden. But the basis
is super important, so this array of numbers only represents a linear transformation as
long as you’re clear what the basis is.
Now I can explain what it means to ‘multiply’ a matrix by a vector. It’s not about multiplication
at all. All it means is you apply the linear transformation that this matrix represents
to the vector this represents. As we saw, this is just equal to this big sum.. Which
you might recognise as the rule for multiplying matrices and vectors.
To see if you’ve got this, see if you can come up with a matrix for the following linear
transformation: It takes the first basis vector and does nothing to it, and takes the second
basis vector and deletes it. Another way of thinking of that is, it takes the second basis
vector to the zero vector- the vector with no length. Pause the video now and try to
come up with the matrix in this basis. Hopefully you’ve had a think. The answer is this.
By the way, your matrices definitely don’t need to be square. The dimensions of the matrix
tell you about the dimension of the vectors it eats vs the dimension of the vectors it
spits out. In this example, this matrix takes 3d vectors,
and gives back 2d ones. Then it’s shape has to be this, because it has 3 rows, representing
what the matrix does to the 3 basis vectors of the initial space, and 2 columns because
each basis vector becomes a 2d vector.
If you’ve following everything so far, then matrix multiplication is going to be really
simple to explain. You know how when you first meet matrix multiplication, they throw this
crazy formula at you. Then they expect you to act shocked when you find out, if you call
this insane thing multiplication, then A times B doesn’t equal B times A. But there is
a much much easier way to understand matrix multiplication. Let’s look at an example
first.
Imagine if you want to transform some vectors. First you want to rotate them by 90 degrees
in the plane. Then you want to project them onto this line like we did earlier. Both of
these are linear, so call the matrix representing these transformations in some basis A and
B. Here’s a question. Is it the same thing to do the projection first, then the rotation?
I can tell you, it’s not, but I want you to pause the video now and think of a vector
that would come out different if you do the projection first or if you do it second. If
you could think of one, say so in the poll here.
Ok, so there are many options but here’s a vector that works: If the vector starts
on the line, and you apply A it gets rotated to here, then when you apply B to the result
you project it down, but in this case, nothing happens to it. But if you did it in the other
order when you apply B it immediately becomes the zero vector, and then when you apply A
nothing else happens. So clearly doing A then B is different to doing B then A
So here’s the big secret: When we write B A, the linear transformation that we’re
talking about is the one where you do A first and then B. That’s all it means. And as
we’ve just seen, order matters, so this is different from the linear transformation
A B, since in that case you swap the order. So the statement AB doesn’t equal BA is
actually blindingly obvious, once you understand the meaning of these things and really shouldn’t
be presented as a startling fact.
So why does this lovely idea of composing maps lead to the awfulness that is the matrix
multiplication formula? Well actually, once you understand it, the formula isn’t really
ugly at all. Let me show you what I means for the case when A and B are 2x2 matrices.
How would we try and write AB as a matrix in a particular basis? Well, let’s follow
our own rules. We need to find out what AB does to the each of the basis vectors. Let’s
do the first one. Well, when we apply B, we get this first column of B, by definition.
But then when we apply A, we get b_11 times the first column of A plus b_21 times the
second column. This result is our first column of AB. Do the same for the second basis vector,
and here is the formula. See? Matrices aren’t so painful, after all.
OK, It’s homework time!
First, Not all sized matrices can be multiplied together. Think about it in terms of them
representing transformations from one space to another, and figure out which size matrices
can be multiplied and explain why in the comments.
Second, and this is a multiple choice one, consider a transformation that takes a 3d
vector, and adds some fixed vector k to it. Say k is the vector (7 3 3). Is this a linear
transformation or not?
Third, imagine you have a matrix A that multiplies the first basis vector by 2, and the second
basis vector by 6. How do you write A in this basis?
As you will have noticed, these questions are from Brilliant.org who are sponsoring
this video. I said this last time, but I think linear algebra is just a topic you can’t
understand without doing examples. If you try and understand the topic just through
learning a bunch of formulas or theorems you just won’t be able to get all the subtleties
involved. I found that when I went through Brilliant’s course, which has really well
designed examples, I remembered a bunch of little tricky points that I’d forgotten
about, and it got me really comfortable with doing actually doing calculations again- since
I don’t do a lot of that anymore.. You can sign up to either a monthly or yearly membership
and access this course and many others on their website. If you follow the link on the
screen or in the description you can get 20% off the yearly membership.
Thanks for watching these linear algebra videos, it’s one of my favourite fields of maths
so I love teaching it because it feels like I’m learning all again myself. Anyway, the
next one will be linked here in 2 weeks so subscribe if you’d like to see it when it’s
out.