[Music]
traditionally dot products are something
that's introduced really early on in a
linear algebra course typically right at
the start so it might seem strange that
I've pushed them back this far in the
series I did this because there's a
standard way to introduce the topic
which requires nothing more than a basic
understanding of vectors but a fuller
understanding of the role the dot
products play in math can only really be
found under the light of linear
transformations before that though let
me just briefly cover the standard way
that dot products are introduced which
I'm assuming is at least partially
review for a number of viewers
numerically if you have two vectors of
the same dimension to lists of numbers
with the same lengths taking their dot
product means pairing up all of the
coordinates multiplying those pairs
together and adding the result so the
vector one two dotted with three four
would be one times three plus two times
four the vector six two eight three
dotted with one eight five three would
be six times 1 plus two times 8 plus 8
times 5 plus 3 times 3 luckily this
computation has a really nice geometric
interpretation to think about the dot
product between two vectors V and W
imagine projecting W onto the line that
passes through the origin and the tip of
V multiplying the length of this
projection by the length of V you have
the dot product V W except when this
projection of W is pointing in the
opposite direction from V that dot
product will actually be negative
so when two vectors are generally
pointing in the same direction their dot
product is positive
when they're perpendicular meaning the
projection of one on to the other is the
zero vector their dot product is zero
and if they point in generally the
opposite direction their dot product is
negative
now this interpretation is weirdly
asymmetric it treats the two vectors
very differently so when I first learned
this I was surprised that order doesn't
matter you could instead project V onto
W multiply the length of the projected V
by the length of W and get the same
result
I mean doesn't that feel like a really
different process
here's the intuition for why order
doesn't matter if V and W happened to
have the same length we could leverage
some symmetry
since projecting W onto V then
multiplying the length of that
projection by the length of V is a
complete mirror image of projecting V
onto W then multiplying the length of
that projection by the length of W
now if you scale one of them say V by
some constant like two so that they
don't have equal length the symmetry is
broken but let's think through how to
interpret the dot product between this
new vector two times V and W if you
think of W is getting projected onto V
then the dot product to V dot W will be
exactly twice the dot product V dot W
this is because when you scale V by two
it doesn't change the length of the
projection of W but it doubles the
length of the vector that you're
projecting onto but on the other hand
let's say you were thinking about V
getting projected onto W well in that
case the length of the projection is the
thing that gets scaled when we multiply
V by two but the length of the vector
that you're projecting onto stays
constant so the overall effect is still
to just double the dot product so even
though symmetry is broken in this case
the effect that this scaling has on the
value of the dot product is the same
under both interpretations
there's also one other big question that
confused me when I first learned this
stuff why on earth does this numerical
process of matching coordinates
multiplying pairs and adding them
together have anything to do with
projection
well they give a satisfactory answer and
also to do full justice to the
significance of the dot product we need
to unearth something a little bit deeper
going on here which often goes by the
name duality but before getting into
that I need to spend some time talking
about linear transformations from
multiple dimensions to one dimension
which is just the number line
these are functions that take in a 2d
vector and spit out some number but
linear transformations are of course
much more restricted than your
run-of-the-mill function with a 2d input
and a 1d output as with transformations
in higher dimensions like the ones I
talked about in Chapter three
there are some formal properties that
make these functions linear but I'm
going to purposefully ignore those here
so as to not distract from our end goal
and instead focus on a certain visual
property that's equivalent to all the
formal stuff if you take a line of
evenly spaced dots and apply a
transformation a linear transformation
will keep those dots evenly spaced once
they land in the output space which is
the number line otherwise if there's
some line of dots that gets unevenly
spaced then your transformation is not
linear
as with the cases we've seen before one
of these linear transformations is
completely determined by where it takes
I hat and J hat but this time each one
of those basis vectors just lands on a
number so when we record where they land
as the columns of a matrix each of those
columns just has a single number
this is a one by two matrix
let's walk through an example of what it
means to apply one of these
transformations to a vector let's say
you have a linear transformation that
takes I hat to 1 and J hat to negative 2
to follow where a vector with
coordinates say for 3 ends up think of
breaking up this vector as 4 times I hat
plus 3 times J hat a consequence of
linearity is that after the
transformation the vector will be 4
times the place where I hat lands 1 plus
3 times the place where J hat lands
negative 2 which in this case implies
that it lands on negative 2
[Music]
when you do this calculation purely
numerically its matrix vector
multiplication
now this numerical operation of
multiplying a 1 by 2 matrix by a vector
feels just like taking the dot product
of two vectors doesn't that 1 by 2
matrix just look like a vector that we
tipped on its side in fact we could say
right now that there's a nice
association between 1 by 2 matrices and
2d vectors defined by tilting the
numerical representation of a vector on
its side to get the Associated matrix or
to tip the matrix back up to get the
Associated vector since we're just
looking at numerical expressions right
now going back and forth between vectors
and one by two matrices might feel like
a silly thing to do but this suggests
something that's truly awesome from the
geometric view there's some kind of
connection between linear
transformations that take vectors to
numbers and vectors themselves
let me show an example that clarifies
the significance and which just so
happens to also enter the dot product
puzzle from earlier unlearn what you
have learned and imagine that you don't
already know that the dot product
relates to projection
what I'm going to do here is take a copy
of the number line and place it
diagonally in space somehow with the
number zero sitting at the origin now
think of the two-dimensional unit vector
whose tip sits where the number one on
the number line is I want to give that
guy a name you had this little guy plays
an important role in what's about to
happen so just keep them in the back of
your mind if we project 2d vector x'
straight onto this diagonal number line
in effect we've just defined a function
that takes 2d vectors to numbers what's
more this function is actually linear
since it passes our visual test that any
line of evenly spaced dots remains
evenly spaced once it lands on the
number line
just to be clear even though I've
embedded the number line in 2d space
like this the outputs of the function
are numbers not 2d vector z' you should
think of a function that takes in two
coordinates and outputs a single
coordinate but that vector U hat is a
two-dimensional vector living in the
input space
it's just situated in such a way that
overlaps with the embedding of the
number line with this projection we just
defined a linear transformation from 2d
vectors to numbers so we're going to be
able to find some kind of 1 by 2 matrix
that describes that transformation to
find that 1 by 2 matrix let's zoom in on
this diagonal number line setup and
think about where I hat and J hat each
land since those landing spots are going
to be the columns of the matrix this
part's super cool we can reason through
it with a really elegant piece of
symmetry since I hat and u hat are both
unit vectors projecting I hat onto the
line passing through u hat looks totally
symmetric to projecting u hat onto the
x-axis so when we ask what number does I
hat land on when it gets projected the
answer is going to be the same as
whatever u hat lands on when it's
projected onto the x-axis but projecting
u hat onto the x-axis just means taking
the x-coordinate of U hat so by symmetry
the number where I hat lands when it's
projected onto that diagonal number line
is going to be the x coordinate of U hat
isn't that cool
the reasoning is almost identical for
the j-hat case think about it for a
moment
for all the same reasons the
y-coordinate of you hat gives us the
number where J hat lands when it's
projected onto the number line copy
pause and ponder that for a moment I
just think that's really cool so the
entries of the 1 by 2 matrix describing
the projection transformation are going
to be the coordinates of U hat and
computing this projection transformation
for arbitrary vectors in space which
requires multiplying that matrix by
those vectors is computationally
identical to taking a dot product with u
hat
this is why taking the dot product with
a unit vector can be interpreted as
projecting a vector onto the span of
that unit vector and taking the length
so what about non unit vectors for
example let's say we take that unit
vector u hat but we scale it up by a
factor of three numerically each of its
components gets multiplied by three so
looking at the matrix associated with
that vector it takes AI hat and J hat to
three times the values where they landed
before
since this is all linear it implies more
generally that the new matrix can be
interpreted as projecting any vector
onto the number line copy and
multiplying where it lands by three this
is why the dot product with a non unit
vector can be interpreted as first
projecting onto that vector then scaling
up the length of that projection by the
length of the vector
take a moment to think about what
happened here we had a linear
transformation from 2d space to the
number line
which was not defined in terms of
numerical vectors or numerical dot
products
it was just defined by projecting space
onto a diagonal copy of the number line
but because the transformation is linear
it was necessarily described by some 1
by 2 matrix and since multiplying a 1 by
2 matrix by a 2d vector is the same as
turning that matrix on its side and
taking a dot product this transformation
was inescapably related to some 2d
vector the lesson here is that anytime
you have one of these linear
transformations whose output space is
the number line no matter how it was
defined there's going to be some unique
vector V corresponding to that
transformation in the sense that
applying the transformation is the same
thing as taking a dot product with that
vector
to me this is utterly beautiful it's an
example of something in math called
duality duality shows up in many
different ways and forms throughout math
and it's super tricky to actually define
loosely speaking it refers to situations
where you have a natural but surprising
correspondence between two types of
mathematical thing for the linear
algebra case that you just learned about
you'd say that the dual of a vector is
the linear transformation that it
encodes and the dual of a linear
transformation from some space to one
dimension is a certain vector in that
space
so to sum up on the surface the dot
product is a very useful geometric tool
for understanding projections and for
testing whether or not vectors tend to
point in the same direction and that's
probably the most important thing for
you to remember about the dot product
but at a deeper level dotting two
vectors together is a way to translate
one of them into the world of
transformations again numerically this
might feel like a silly point to
emphasize it's just two computations
that happen to look similar but the
reason I find this so important is that
throughout math when you're dealing with
a vector once you really get to know
it's personality sometimes you realize
that it's easier to understand it not as
an arrow in space but as the physical
embodiment of a linear transformation
it's as if the vector is really just a
conceptual shorthand for a certain
transformation since it's easier for us
to think about arrows in space rather
than moving all of that space to the
number line in the next video you'll see
another really cool example of this
duality in action as I talk about the
cross product
[Music]