Welcome to a lesson
on how to determine the singular value decomposition
of a given matrix.
Let A be an m by n matrix.
Then A equals U times Sigma times V transpose
is the singular value decomposition of A.
U is an m by m orthogonal matrix with columns equal
to the unit eigenvectors of A times A transpose.
V is an n by n orthogonal matrix
whose columns are unit eigenvectors of A transpose times A.
Just remember, we're looking for V transpose
for the singular value decomposition.
And Sigma is an m by n matrix with the singular values
of matrix A on the main diagonal
and all other entries of zero.
We will look at an example
where matrix A is a 2 by 3 matrix.
So let's check the dimensions of U, V,
as well as V transpose and Sigma.
If A is a 2 by 3 matrix, m is 2 and n is 3,
and therefore U is a 2 by 2 matrix.
Sigma is m by n matrix
and, therefore, Sigma is a 2 by 3 matrix.
And V transpose has the same dimensions as matrix V,
which is an n by n matrix,
V transpose is a 3 by 3 matrix.
And now let's look at an example.
To find the singular value decomposition
of the given matrix,
we will follow the 3 steps shown on the right.
Number 1, we'll determine matrix V
and then the matrix V transpose.
Number 2, we'll determine the singular values of matrix A
and then form the matrix Sigma.
And then step 3, we'll determine matrix U by using formulas,
which we'll discuss more later.
So because matrix V is an n by n orthogonal matrix
whose columns are unit eigenvectors of A transpose times A,
we'll begin by determining A transpose times A.
To find the matrix A transpose,
the first row of matrix A becomes the first column
of A transpose.
The second row of matrix A becomes
the second column of A transpose.
A transpose times A results in the 3 by 3 matrix
shown here on the right.
You may want to pause the video and check this product.
Because we're looking for unit eigenvectors
of this 3 by 3 matrix,
the next step is to determine the eigenvalues.
We determine the eigenvalues by solving the equation
the determinant of the difference of matrix A and lambda
times the identity matrix equals zero,
where matrix A is a 3 by 3 matrix resulting
from A transpose times A.
Simplifying inside the parentheses,
we now need to find the determinant of this 3 by 3 matrix
and set it equal to zero.
I will go ahead and assume you can find this determinant
which will give us the characteristic equation.
The factored form of the characteristic equation
is lambda times the quantity lambda minus 16
times the quantity lambda minus 6, which must equal zero.
Giving the eigenvalues from greatest to least,
we have lambda sub 1 equals 16,
lambda sub 2 equals 6,
and lambda sub 3 equals zero.
The next step is to find corresponding unit eigenvectors,
which will give us the columns of matrix V
and then we can find V transpose.
Beginning with lambda sub 1 equals 16,
to find the corresponding eigenvectors,
we need to solve the vector equation
the difference of matrix A and lambda
times the identity matrix times vector x
equals a zero vector where vector x is an eigenvector
and matrix A is the result of A transpose times A.
I've already set up the vector equation.
For the next step we write the matrix equation.
To find the coefficient matrix here,
notice how we subtracted 16 along the main diagonal
of the 3 by 3 matrix we found
by multiplying A transpose and A.
Next, we write the augmented matrix
and write the augmented matrix in reduced row echelon form.
In reduced row echelon form,
notice how the first row indicates x1 minus x3 equals zero.
The second row indicates x2 equals zero.
x3 is a free variable. We have x3 equals x3.
If we let x3 equal t,
the corresponding eigenvectors are in the form
of t times the vector 1, 0, 1.
If we let t equal 1,
we would have the eigenvector 1, 0, 1,
which we need to normalize to find vector v sub 1,
which will be the first column of matrix V.
The magnitude of the vector 1, 0, 1 is the square root of 2
and therefore the vector v1 is equal
to the vector with components 1
divided by the square root of 2,
0, 1 divided by the square root of 2.
Again, this will be the first column of matrix V.
And now we need to go through the same process
for lambda sub 2 equals 6 and lambda sub 3 equals zero.
So for lambda sub 2 equals 6 we have our vector equation,
the matrix equation, the augmented matrix,
and the augmented matrix in reduced row echelon form.
Row 1 indicates that x1 plus x3 equals zero,
row 2 indicates x2 plus x3 equals zero.
And once again, x3 is a free variable.
Letting x3 equal t,
the corresponding eigenvectors are in the form
of t times the vector -1, -1, 1.
Again letting t equal 1,
we have the eigenvector of -1, -1, 1,
which we need to normalize to find the vector v sub 2.
The magnitude of the vector -1, -1, 1
is the square root of 3,
and, therefore, the vector v sub 2,
which will be the second column of matrix V,
has components -1 divided by the square root of 3,
1 divided by the square root of 3
and 1 divided by the square root of 3.
And, finally, we have lambda sub 3 equals zero.
Here we have our vector equation,
our matrix equation, the augmented matrix,
and the augmented matrix in reduced row echelon form.
First row indicates x1 plus x3 equals zero,
second row indicates x2 minus 2x3 equals zero.
And again, x3 is a free variable.
Letting x3 equal t,
we have the corresponding eigenvectors
in the form of t times the vector -1, 2, 1.
Again letting t equal 1 and normalizing the eigenvector,
the magnitude of the vector -1, 2, 1 is the square root of 6
and, therefore, vector v sub 3 is the vector with components
1 divided by the square root of 6,
2 divided by the square root of 6,
and 1 divided by the square root of 6.
This will be the third column of matrix V.
So again,
now that we have vector v1, vector v2, and vector v3,
we can form matrix V
where these vectors form the columns of matrix V.
And again, for the singular value decomposition,
we need V transpose.
Making the rows of matrix V the columns of V transpose,
we now have the vector V transpose.
Now we'll determine the singular values of matrix A,
which will be the entries along the main diagonal
of the matrix Sigma and all their entries will be zero.
The singular values of matrix A are the square roots
of only the positive eigenvalues
of either A transpose times A or A times A transpose.
Both of these products have the same positive eigenvalues.
Well, we already found the eigenvalues
of A transpose times A,
which were lambda sub 1, lambda sub 2 and lambda sub 3.
We do not include the zero though.
We only use the positive eigenvalues
to determine the singular values.
And the singular values are equal to the square roots
of the positive eigenvalues.
We indicate singular values by using the lowercase sigma.
In this case, sigma sub 1 and sigma sub 2,
which again need to be given in order
from greatest to least.
And, therefore, the first singular value, sigma sub 1,
is equal to the square root of lambda sub 1,
which equals the square root of 16, or 4.
lambda sub 2 is equal to the square root of lambda sub 2,
which equals the square root of 6.
And now we use these values
along the main diagonal of matrix U,
which is a 2 by 3 matrix.
All other entries are zero.
And now we have the matrix Sigma.
The last step is to find matrix U,
which we will do using formulas.
We know for the singular value decomposition,
we have A equals U times Sigma times V transpose.
If we take this equation and multiply
both sides by matrix V,
V transpose times V is equal to the identity matrix
because V and V transpose are orthogonal,
which gives us the equation A times V equals U times Sigma.
And if we work this out, it'll give us the formulas below
that we can use to find the vector u sub 1
and vector u sub 2.
It works out that matrix A times the vector v sub 1
is equal to sigma sub 1 times the vector u sub 1
and, similarly, matrix A times vector v sub 2
is equal to sigma sub 2 times the vector u sub 2.
From here we can find vector u sub 1 and vector u sub 2.
Solving the first equation for vector u sub 1,
we would divide both sides by sigma sub 1,
which gives us vector u sub 1
equals 1 divided by sigma sub 1 times matrix A
times the vector v sub 1,
which I've already written out here.
The result is that vector u sub 1
with components 1 divided by the square root of 2,
1 divided by the square root of 2.
And using the second equation,
we can solve for vector u sub 2.
Vector u sub 2 equals 1 divided by sigma sub 2
times matrix A times vector v sub 2,
which, again, I've already written out.
The result is that vector u sub 2,
which has components 1 divided by the square root of 2,
1 divide by the square root of 2.
And this is what we need to find the matrix U.
These two vectors form the columns of matrix U,
which we have here.
Now, you might be asking, why do we use these formulas
rather than find the unit eigenvectors
of A times A transpose?
Well, remember, when it comes to these unit eigenvectors,
the first formula they gave us,
vector u sub 1 equal to 1 divided by the square root of 2,
1 divided by the square root of 2,
but there's also a unit vector in the opposite direction,
which, if we used,
would not give us an accurate singular value decomposition.
But by using the formulas,
we will get the correct unit eigenvector.
So to finish this off,
we now know A is equal to U times Sigma times V transpose,
which I've written out here,
which I also checked using the Desmos matrix calculator
and the product on the right is equal to matrix A.
I hope you found this helpful.