Professor Dave here, let’s talk about orthogonality.
We’ve already touched on orthogonality during our lesson about the vector dot product, and
we may simply think of orthogonal vectors as ones that are perpendicular.
Of course that is true, but now it’s time to expand on this basic understanding, so
let’s go in for a closer look.
First, let’s recall the basic definition we just mentioned, whereby two vectors in
a vector space are considered orthogonal if they are perpendicular to one another, which
means that the angle between them is half pi radians, or ninety degrees.
This situation will occur if the dot product between the two vectors is equal to zero,
since the dot product is equal to the product of the two vector lengths, times the cosine
of the angle between them, and the cosine of half pi is zero, which makes the dot product
equal to zero.
Let’s now consider the vectors a = (4, 2, -1) and b = (1, -3, -2).
To see if these vectors are orthogonal, we must check that the dot product is zero.
To take the vector dot product, apart from the geometric definition we already mentioned,
we can also use this algebraic one, where we find the product of the first components,
plus the product of the second components, plus the product of the third components.
In this particular case, this makes a dot b equal to (4 times 1) + (2 times -3) + (-1
times -2), which is 4 – 6 + 2, or zero.
Since the dot product is equal to zero, our vectors are indeed orthogonal.
Now let’s expand on this basic concept.
In addition to vectors being orthogonal, we can use another restriction on some vectors
to create another useful concept, orthonormality.
The “normal” part here refers to the vectors having a length of 1.
If a set of vectors are all of length 1 and also orthogonal, they are said to be orthonormal.
We have already seen that one way to get the length of a vector is to take the square root
of the dot product involving the vector and itself, meaning the length of vector A is
equal to the square root of A dot A. Now, if we then divide the original vector by that
length, we will get a new vector that will be of length 1.
With that understood, let’s take our two vectors from before, which were a = (4, 2,
-1) and b = (1, -3, -2).
The length of A will be given by the square root of 42 + 22 + (-1)2, which gives us the
square root of 16 + 4 + 1, which is equal to root 21.
The length of B will be given by the square root of 12 + (-3)2 + (-2)2 which gives us
the square root of 1 + 9 + 4, which is equal to root 14.
Now we can divide the vectors A and B by the lengths we just calculated, and we get new
“unit” vectors (4, 2, -1) over root 21, and (1, -3, -2) over root 14.
Here, the word “unit” means that they have lengths that are equal to one.
This process of making vectors into unit vectors is often called “normalization”.
By doing this we have only changed the magnitude of the vectors, and not the direction.
Because of this, they are still orthogonal as we previously verified.
And since these two vectors are orthogonal and also have lengths equal to 1, that makes
them orthonormal as well.
It is actually not the case that only vectors can be considered orthogonal, we can also
have orthogonal subspaces.
Two subspaces, A and B, are said to be orthogonal if every vector in A is orthogonal to every
vector in B. So for any vector a in subspace A, and any vector b in subspace B, a dot b
must be equal to zero.
Consider the following subspaces of R3.
Subspace A is made of the vectors of the form (a1, 0, 0), which can have any value as the
first component but must have zeros as the second and third.
Subspace B is made of the vectors of the form (0, b2, b3), which have a zero as the first
component and can have any value as the second and third.
While there are an infinite number of vectors that can satisfy these conditions, taking
the dot product of any possible vector in subspace A and any possible vector in subspace
B gives us (a1 times zero) + (zero times b2) + (zero times b3), and this will be equal
to zero no matter what those values are.
Because of this, we have shown that the subspaces A and B are orthogonal.
Square matrices can also be orthogonal.
The condition for this is simply that the columns of the matrix make up an orthonormal set.
Take for example the 2 by 2 matrix made up of 1 over root 2 in all elements, but the
bottom right is negative.
Taking the columns of this matrix as vectors, (1 over root 2, 1 over root 2) and (1 over
root 2, -1 over root 2), let’s verify that they are orthonormal.
First to check for orthogonality, we can take the dot product, which will give us (1 over
root 2 times 1 over root 2) + (1 over root 2 times -1 over root 2).
This becomes one half minus one half, which is simply zero, meaning that these vectors
are orthogonal.
Now we just need to verify that they both have lengths of 1.
The length of the first will be the square root of (1 over root 2 quantity squared) +
(1 over root 2 quantity squared), which is the square root of one half plus one half,
or the square root of 1, which is 1.
Then for the second, the length is the square root of (1 over root 2 quantity squared) +
(-1 over root 2 quantity squared), which again becomes the square root of one half plus one
half, or the square root of 1, which is 1.
Both vectors have a length of 1, and are orthogonal to each other.
This makes the columns of the matrix an orthonormal set, meaning that the matrix itself is orthogonal.
The main benefit to having an orthogonal matrix is that the inverse of the matrix is simply
the same as its “transpose”.
By transpose, we mean that the columns of the matrix become the rows and vice versa.
For example, the transpose of a 2 by 2 matrix and a 3 by 3 matrix are given as shown.
You can see that the first row becomes the first column, the second row becomes the second
column, and so on.
For an orthogonal matrix, generating the inverse is as simple as finding the transpose, and
this makes it extremely easy to find the inverse of orthogonal matrices, as you simply swap
some elements around.
Lastly, we can even consider orthogonality between functions.
However to do this, we must consider a definition for something called the “inner product”
from point a to point b for the functions f(x) and g(x).
This inner product, written as f comma g between angular brackets, is defined to be the integral,
from a to b, of f(x) times g(x), dx.
Once again, if this inner product ends up being equal to zero, the functions are said
to be orthogonal.
However, in this case there is some dependence on what range of values we are considering.
Take for example the functions f(x) = x and g(x) = 1.
Our inner product will be equal to the integral from a to b, of x dx.
As we recall from our study of calculus, the antiderivative here will be x2/2, and we must
evaluate this at b, and then subtract from that the evaluation at a.
Let’s say our interval goes from a = -1 to b = 1.
Then our inner product becomes 12/2 - (-1)2/2, which is equal to zero.
So our functions are orthogonal across this range.
However, if we consider the range from a = 0 to b = 1, then our inner product is now 12/2
- 02/2, which is equal to 1/2, and therefore no longer zero.
So our functions are not orthogonal over this range of values.
There are many things to consider when looking at the inner product of two functions.
In fact, often times an extra function, w(x), referred to as the weight function, is added
to the definition of the inner product so that equals the integral from a to
b of f(x) times g(x) times w(x) dx.
This weight function can be any of a variety of different things, meaning we can have functions
that are orthogonal with respect to a variety of different functions.
Orthogonality ends up playing a key role in several aspects of math and even science.
It helps us break down systems into distinct elements, for easier understanding and problem solving.
So to make sure we understand, let’s check comprehension.