[Music]
you
as you can probably tell by now the bulk
of this series is on understanding
matrix and vector operations through
that more visual lens of linear
transformations this video is no
exception
describing the concept of inverse
matrices column space rank and null
space through that lens a forewarning
though I'm not going to talk about the
methods for actually computing these
things and some would argue that that's
pretty important there are a lot of very
good resources for learning those
methods outside this series keywords
Gaussian elimination and row echelon
form I think most of the value that I
actually have to add here is on the
intuition 1/2 plus in practice we
usually get software to compute this
stuff for us anyway first a few words on
the usefulness of linear algebra by now
you already have a hint for how it's
used in describing the manipulation of
space which is useful for things like
computer graphics and robotics but one
of the main reasons that linear algebra
is more broadly applicable and required
for just about any technical discipline
is that it lets us solve certain systems
of equations when I say system of
equations I mean you have a list of
variables things you don't know and a
list of equations relating them in a lot
of situations those equations can get
very complicated but if you're lucky
they might take on a certain special
form within each equation the only thing
happening to each variable is that it's
scaled by some constant and the only
thing happening to each of those scaled
variables is that they're added to each
other so no exponents or fancy functions
are multiplying two variables together
things like that the typical way to
organize this sort of special system of
equations is to throw all the variables
on the left
and put any lingering constants on the
right
it's also nice to vertically line up the
common variables and to do that you
might need to throw in some zero
coefficients whenever the variable
doesn't show up in one of the equations
this is called a linear system of
equations you might notice that this
looks a lot like matrix vector
multiplication in fact you can package
all of the equations together into a
single vector equation where you have
the matrix containing all of the
constant coefficients and a vector
containing all of the variables and
their matrix vector product equals some
different constant vector let's name
that constant matrix a denote the vector
holding the variables with a bold-faced
X and call the constant vector on the
right-hand side V this is more than just
a notational trick to get our system of
equations written on one line it sheds
light on a pretty cool geometric
interpretation for the problem the
matrix a corresponds with some linear
transformation so solving ax equals V
means we're looking for a vector X which
after applying the transformation lands
on V
think about what's happening here for a
moment you can hold in your head this
really complicated idea of multiple
variables all intermingling with each
other just by thinking about squishing
and morphing space and trying to figure
out which vector lands on another cool
right to start simple let's say you have
a system with two equations and two
unknowns this means the matrix a is a 2
by 2 matrix and V and X are each 2
dimensional vectors now how we think
about the solutions to this equation
depends on whether the transformation
associated with a squishes all of space
into a lower dimension like a line or a
point or if it leaves everything
spanning the full two dimensions where
it started
in the language of the last video we
subdivide into the cases where a has
zero determinant and the case where a
has nonzero determinant
let's start with the most likely case
where the determinant is nonzero meaning
space does not get squished into a zero
area region in this case there will
always be one and only one vector that
lands on V and you can find it by
playing the transformation in Reverse
following where V goes as we rewind the
tape like this you'll find the vector X
such that a times x equals V
when you play the transformation in
Reverse it actually corresponds to a
separate linear transformation commonly
called the inverse of a denoted a to the
negative one for example if a was a
counterclockwise rotation by 90 degrees
then the inverse of a would be a
clockwise rotation by 90 degrees
if a was a rightward shear that pushes
j-hat one unit to the right the inverse
of a would be a leftward shear that
pushes j-hat one unit to the left in
general a inverse is the unique
transformation with the property that if
you first apply a then follow it with
the transformation a inverse you end up
back where you started applying one
transformation after another is captured
algebraically with matrix multiplication
so the core property of this
transformation a inverse is that a
inverse times a equals the matrix that
corresponds to doing nothing the
transformation that does nothing is
called the identity transformation it
leaves I hat and J hat each where they
are unmoved so its columns are 1 0 and 0
1
once you find this inverse which in
practice you do with a computer you can
solve your equation by multiplying this
inverse matrix by V
and again what this means geometrically
is that you're playing the
transformation in Reverse and following
V
this nonzero determinant case which for
a random choice of matrix is by far the
most likely one corresponds with the
idea that if you have two unknowns and
two equations it's almost certainly the
case that there's a single unique
solution this idea also makes sense in
higher dimensions when the number of
equations equals the number of unknowns
again the system of equations can be
translated to the geometric
interpretation where you have some
transformation a
and some vector V and you're looking for
the vector X that lands on V
as long as the transformation a doesn't
squish all of space into a lower
dimension meaning its determinant is
nonzero
there will be an inverse transformation
a inverse with the property that if you
first do a then you do a inverse it's
the same as doing nothing
and to solve your equation you just have
to multiply that reverse transformation
matrix by the vector V
but when the determinant is zero and the
transformation associated with the
system of equations squishes space into
a smaller dimension there is no inverse
you cannot unscrew a line to turn it
into a plane at least that's not
something that a function can do that
would require transforming each
individual vector into a whole line full
of vectors but functions can only take a
single input to a single output
similarly for three equations and three
unknowns there will be no inverse if the
corresponding transformation squishes 3d
space onto the plane or even if it
squishes it on to a line or a point
those all correspond to a determinant of
zero since any region is squished into
something with zero volume
it's still possible that a solution
exists even when there is no inverse
it's just that when your transformation
squishes space on to say a line you have
to be lucky enough that the vector V
lives somewhere on that line
you might notice that some of these zero
determinant cases feel a lot more
restrictive than others given a 3x3
matrix for example it seems a lot harder
for a solution to exist when it squishes
space onto a line compared to when it
squishes things onto a plane even though
both of those are zero determinant
we have some language that's a bit more
specific than just saying zero
determinant when the output of a
transformation is a line meaning it's
one dimensional we say the
transformation has a rank of 1
if all the vectors land on some
two-dimensional plane we say the
transformation has a rank of 2
so the word rank means the number of
dimensions in the output of a
transformation for instance in the case
of 2 by 2 matrices Rank 2 is the best
that it can be
it means the basis vectors continue to
span the full two dimensions of space
and the determinant is not zero but for
3x3 matrices rank two means that we've
collapsed but not as much as they would
have collapsed for a rank one situation
if a 3d transformation has a nonzero
determinant and its output fills all of
3d space it has a rank of three this set
of all possible outputs for your matrix
whether it's a line a plane 3d space
whatever is called the column space of
your matrix you can probably guess where
that name comes from the columns of your
matrix tell you where the basis vectors
land and the span of those transformed
basis vectors gives you all possible
outputs in other words the column space
is the span of the columns of your
matrix
so a more precise definition of rank
would be that it's the number of
dimensions in the column space
when this rank is as high as it can be
meaning it equals the number of columns
we call the matrix full rank
notice the zero vector will always be
included in the column space since
linear transformations must keep the
origin fixed in place for a full rank
transformation the only vector that
lands at the origin is the zero vector
itself but for matrices that aren't full
rank which squish to a smaller dimension
you can have a whole bunch of vectors
land on zero
if a 2d transformation squishes space
onto a line for example there is a
separate line in a different direction
full of vectors that get squashed onto
the origin if a 3d transformation
squishes based onto a plane there's also
a full line of vectors that land on the
origin
if a 3d transformation squishes all the
space onto a line then there's a whole
plane full of vectors that land on the
origin
this set of vectors that lands on the
origin is called the null space or the
kernel of your matrix it's the space of
all vectors that become null in the
sense that they land on the zero vector
in terms of the linear system of
equations when V happens to be the zero
vector the null space gives you all of
the possible solutions to the equation
so that's a very high-level overview of
how to think about linear systems of
equations geometrically each system has
some kind of linear transformation
associated with it and when that
transformation has an inverse you can
use that inverse to solve your system
otherwise the idea of columnspace lets
us understand when a solution even
exists and the idea of a null-space
helps us to understand what the set of
all possible solutions can look like
again there's a lot that I haven't
covered here most notably how to compute
these things I also have to limit my
scope to examples where the number of
equations equals the number of unknowns
but the goal here is not to try to teach
everything it's that you come away with
a strong intuition for inverse matrices
column space and null space and that
those intuitions make any future
learning that you do more fruitful next
video by popular request will be a brief
footnote about non square matrices then
after that I'm going to give you my take
on dot products and something pretty
cool that happens when you view them
under the light of linear
transformations see you then
[Music]