Let's apply this to face recognition and it's referred to as using Eigenfaces and
it comes from this paper by Matt Turk and Alex Pentland, Sandy Pentland.
It's Alex Pentland, but his name is, he goes by Sandy.
They were at the MIT media lab at the time.
And one of the reasons this paper, besides the fact that it works so
well, is just known so
well, in my humble opinion, is because of this term, Eigenfaces and face space.
What a great, what a great phrase.
In fact, I can show you the Eigenfaces, I will in just a minute.
All right.
So, here's what they do.
They assume that most face images are going to lie on some
low-dimensional subspace in the big image space.
Determine by, I don't know, let's say, k eigenvectors.
Okay? K directions of maximum variance where k
is going to be way smaller than d.
So, d might be 10,000 or a million.
K is going to be like 20 or 200.
Both of those numbers are way smaller than 10,000 or a million.
So, what it does is they use PCA, like I just showed you,
to find the vectors, or what are called the Eigenfaces.
We'll look at them in a minute, u1 through uk, that span, that subspace.
Okay? So, you take all your images,
you find your eigenvectors.
And now, what you're going to do, and this is the really cool part,
is you're going to represent your face images in that data set
as just the linear combination of those eigenvectors.
Or another way of saying it is,
I'm just going to have the coefficients of the 20, of the 20 eigenvectors.
If it's 200, 200 eigenvectors.
And I'm going to represent my entire image, all 10,000 numbers,
by just the coefficients of these eigenvectors.
And I multiply those coefficients times the eigenvector, sum them up,
that would be my new image.
All right.
So, it's a tremendous data reduction.
So, take you through an example here.
So, and some of these images come directly from the old papers,
some from some newer work, but makes the same point.
So, suppose I have some training images X1 through XM.
So, here's a picture of faces.
Now, notice, it's just the cropped part of the face.
So, they're going to try to do recognition just on the cropped
part of the face.
So, the first thing that's kind of cool to look at is this is the mean image,
mean face.
Not the mean face, the mean face, the average face.
Now, nobody wants to have the average face.
Too bad.
Whoever that is, he or she has the average face.
Why did I say he or she?
Well, by definition, it's hard to tell.
Right? because it's an average over everybody.
There is some interesting work done on showing how average faces
appear more beautiful than real faces.
This was done a long time ago.
It's just a study of the appearance of beauty, anyway.
It was interesting image processing approach to thinking about,
could you predict when a face would be deemed beautiful.
All right.
You can imagine here's a question.
Show a picture to a computer, face image, part of the digression, and
say on the average, on a scale of one to ten, how likely are people to rate this
thing as beautiful, as this person is beautiful?
And could you do that?
I think these days using machine learning might be a different approach.
But at the time, there was work done looking at average faces.
Sure you wanted to know that.
Anyway, oh, and so what do you do with the mean face?
Like what you should do with a mean face, you throw it away.
Well, what you really do is subtract it out of your population.
Isn't that great?
We subtract all the [LAUGH] mean faces out of the population.
No, we subtract the mean face from all the, and
what we have left is our distribution.
And then, we start computing the eigenvectors.
And here they are. Okay?
So, here's some top eigenvectors, and one of the things you'll notice
right away, fact, let's take a look at this first eigenvector.
Actually, the, the second one.
Is that it's much brighter on one side than the other,
because some of the variation might just be due to the lighting coming from one
side or the other.
But later, you'll have these different eigenvectors,
which look like these kind of ghostly images.
But these are eigenvectors.
So, they look to you like pictures and they are, but what they are is
they're a 10,000 dimensional vector, which is just an image.
Remember, I can go between the image space 10,000, right, and the images, so
I'm just showing you these eigenvectors as images.
Okay? And if I want to take a dot product
of a real picture and these?
That's easy.
All I do is overlay them and multiply.
Multiply every pixel and sum them up.
because that's what a dot product is.
So, when I show this to students,
sometimes they don't understand how come the images are eigenvectors.
And that's because in image space, that's what it means,
a vector is an image.