DAVID SHIROKOFF: Hi everyone.
Welcome back.
So today I'd like to tackle a problem on pseudoinverses.
So given a matrix A, which is not square,
so it's just 1 and 2.
First, what is its pseudoinverse?
So A plus I'm using to denote the pseudoinverse.
Then secondly, compute A plus A and A A plus.
And then thirdly, if x is in the null space of A,
what is A plus A acting on x?
And lastly, if x is in the column space of A transpose,
what is A plus A*x?
So I'll let you think about this problem for a bit,
and I'll be back in a second.
Hi everyone.
Welcome back.
OK, so let's take a look at this problem.
Now first off, what is a pseudoinverse?
Well, we define the pseudoinverse using the SVD.
So in actuality, this is nothing new.
Now, we note that because A is not square,
the regular inverse of A doesn't necessarily exist.
However, we do know that the SVD exists for every matrix A
whether it's square or not.
So how do we compute the SVD of a matrix?
Well let's just recall that the SVD of a matrix
has the form of U sigma V transpose, where U and V are
orthogonal matrices and sigma is a matrix
with positive values along the diagonal
or 0's along the diagonal.
And let's just take a look at the dimensions
of these matrices for a second.
So we know that A is a 1 by 2 matrix.
And the way to figure out what the dimensions
of these matrices are I usually always
start with the center matrix, sigma,
and sigma is always going to have the same dimensions as A,
so it's going to be a 1 by 2 matrix.
U and V are always square matrices.
So to make this multiplication work out,
we need V to have 2, and because it's
square it has to be 2 by 2.
And likewise, U has to be 1 by 1.
So we now have the dimensions of U, sigma, and V.
And note, because U is a 1 by 1 matrix,
the only orthogonal 1 by 1 matrix is just 1.
So u we already know is just going
to be the matrix, the identity matrix, which is a 1
by 1 matrix.
OK, now how do we compute V and sigma?
Well, we can take A transpose and A,
and if we do that, we end up getting the matrix V sigma
transpose sigma V transpose.
And this matrix is going to be a square matrix where
the diagonal elements are squares of the singular values.
So computing V and the values along sigma
just boil down to diagonalizing A transpose A.
So what is A transpose A?
Well, in our case is [1; 2] times [1, 2],
which gives us [1, 2; 2, 4].
And note that the second row is just a constant multiple
times the first row.
Now what this means is we have a zero eigenvalue.
So we already know that lambda_1 is going to be 0.
So one of the eigenvalues of this matrix is 0.
And of course, when we square root it,
this is going to give us a singular value
sigma, which is also 0.
And this is generally a case when we have
a sigma which is not square.
We typically always have zero singular values.
Now to compute the second eigenvalue,
well we already know how to compute
the eigenvalues of a matrix, so I'm just
going to tell you what it is.
The second one is lambda is 5.
And if we just take a quick look what
the corresponding eigenvector is going to be to lambda is 5,
it's going to satisfy this equation.
So we can take the eigenvector u to be 1 and 2.
However, remember that when we compute
the eigenvector for this orthogonal matrix V,
they always have to have a unit length.
And this vector right now doesn't have a unit length.
We have to divide by the length of this vector, which
in our case is 1 over root 5.
And if I go back to the lambda equals 0 case,
we also have another eigenvector,
which I'll just state.
You can actually compute it quite quickly
just by noting that it has to be orthogonal to this eigenvector,
2 and 1.
So what this means is A has a singular value decomposition,
which looks like: 1, so this is u, times sigma,
which is going to be root 5, 0.
Remember that the first sigma is actually the square root
of the eigenvalue.
Times a matrix which looks like, now we
have to order the eigenvalues up in the correct order.
Because 5 appears in the first column,
we have to take this vector to be in the first column as well.
So this is 1 over root 5, this is 2 over root 5, negative 2
over root 5, and 1 over root 5.
And now this is V, but the singular value decomposition
is defined by V transpose.
So this gives us a representation for A. And now
once we have the SVD of A, how do we actually
compute A plus, or the pseudoinverse of A?
Well just note if A was invertible,
then the inverse of A in terms of the SVD
would be V transpose times the inverse of sigma.
Sorry, this is not V transpose, this is just V.
So it'd be V sigma inverse U transpose.
And when A is invertible, sigma inverse exists.
So in our case, sigma inverse doesn't necessarily
exist because sigma-- note, this is
sigma-- sigma is root 5 and 0.
So we have to construct a pseudoinverse for sigma.
So the way that we do that is we take 1
over each singular value, and we take the transpose of sigma.
So when A is not invertible, we can still
construct a pseudoinverse by taking
V, an approximation for sigma inverse, which in our case
is going to be 1 over the singular value and 0.
So note where sigma is invertible,
we take the inverse, and then we fill in 0's in the other areas.
Times U transpose.
And we can work this out.
We get 1 over root 5, 1, minus 2; 2, 1, 1 over root 5, 0.
And if I multiply things out, I get 1/5, [1; 2].
So this is an approximation for A inverse,
which is the pseudoinverse.
So this finishes up part one.
And I'll started on part two in a second.
So now that we've just computed A plus, the pseudoinverse of A.
We're going to investigate some properties
of the pseudoinverse.
So for part two we need to compute
A times A plus and A plus times A.
So we can just go ahead and do this.
So A A plus you can do fairly quickly.
1/5, [1; 2].
And when we multiply it out we get 1 plus 4 divided by 5 is 1.
So we just get the one by one matrix, which
is 1, the identity matrix.
And secondly, if we take A plus times A
we're going to get 1/5, [1; 2] times [1, 2].
And we can just fill in this matrix.
This is 1/5, [1, 2; 2, 1].
And this concludes part two.
So now let's take a look at what happens when a vector x is
in the null space of A, and then secondly,
what happens when x is in the column space of A transpose.
So for part three, let's assume x
is in the null space of A. Well what's the null space of A?
We can quickly check that the null space of A
is a constant times any vector minus 2, 1.
So that's the null space.
So if x is, for example, i.e.
if we take x is equal to minus 2, 1,
and we were to, say, multiply it by A plus A, acting on x,
we see that we get 0.
And this isn't very surprising because, well,
if x is in the null space of A, we know that A acting on x
is going to be 0.
So that no matter what matrix A plus is, when we multiply by 0,
we'll always end up with 0.
And then lastly, let's take a look at the column
space of A transpose.
Well, A transpose is [1, 2], so it's
any constant times the vector [1; 2].
And specifically, if we were to take, say, x is equal to [1;
2], we can work at A plus A acting on the vector [1; 2].
So we have 1/5 [1, 2; 2, 1].
So recall this is A plus A. And if we multiply it
on the vector [1; 2], we get 1 plus 4 is 5, divided by 5,
so we get 1.
2 plus 2 is 4-- sorry, I copied the matrix down.
So it's 2 plus 8, which is 10, divided by 5 is 2.
And we see that at the end we recover the vector x.
So in general, if we take A plus A acting
on x, where x is in the column space of A transpose,
we always recover x at the end of the day.
So intuitively, what does this matrix A plus A do?
Well, if x is in the null space of A, it just kills it.
We just get 0.
If x is not in the null space of A, then we just get x back.
So it's essentially the identity matrix
acting on x whenever x is in the column space of A transpose.
Now specifically, if A is invertible,
then A doesn't have a null space.
So what that means is: when A is invertible,
A plus A recovers the identity because when we multiply it
on any vector, we get that vector back.
So I'd like to conclude here, and I'll see you next time.