Why is linear algebra actually useful? There very many applications of linear algebra.
In data science, in particular, there are several ones of high importance.
Some are easy to grasp, others not just yet. In this lesson, we will explore 3 of them:
• Vectorized code also known as array programming • Image recognition
• Dimensionality reduction Okay. Let’s start from the simplest and
probably the most commonly used one – vectorized code.
We can certainly claim that the price of a house depends on its size. Suppose you know
that the exact relationship for some neighborhood is given by the equation:
Price equals 10,190 + 223 times size. Moreover, you know the sizes of 5 houses 693,
656, 1060, 487, and 1275 square feet. What you want to do is plug-in each size in
the equation and find the price of each house, right?
Well, for the first one we get: 10190 + 223 times 693 equals 164,729.
Then we can find the next one, and so on, until we find all prices.
Now, if we have 100 houses, doing that by hand would be quite tedious, wouldn’t it?
One way to deal with that problem is by creating a loop. You can iterate over the sizes, multiplying
each of them by 223, and adding 10,190. However, we are smarter than that, aren’t
we? We know some linear algebra already. Let’s explore these two objects:
A 5 by 2 matrix and a vector of length 2. The matrix contains a column of 1s and another
– with the sizes of the houses. The vector contains 10,190 and 223 – the numbers from
the equation. If we go about multiplying them, we will get
a vector of length 5. The first element will be equal to:
1 times 10,190 plus 693 times 223. The second to:
1 times 10,190 plus 656 times 223. And so on.
By inspecting these expressions, we quickly realize that the resulting vector contains
all the manual calculations we made earlier to find the prices.
In machine learning and linear regressions in particular, this is exactly how algorithms
work. We’ve got an inputs matrix; a weights, or a coefficients matrix; and an output matrix.
Without diving too deep into the mechanics of it here, let’s note something.
If we have 10,000 inputs, the initial matrix would be 10,000 by 2, right? The weights matrix
would still be 2 by 1. When we multiply them, the resulting output matrix would be 10,000-by-1.
This shows us that no matter the number of inputs, we will get just as many outputs.
Moreover, the equation doesn’t change, as it only contained the two coefficients – 10,190
and 223. Alright.
So, whenever we are using linear algebra to compute many values simultaneously, we call
this ‘array programming’ or ‘vectorized code’. It is important to stress that array
programming is much, much faster. There are libraries such as NumPy that are optimized
for performing this kind of operations which greatly increases the computational efficiency
of our code. Okay.
What about image recognition? In the last few years, deep learning, and
deep neural networks in particular, conquered image recognition. On the forefront are convolutional
neural networks or CNNs in short. What the basic idea? You can take a photo, feed it
to the algorithm and classify it. Famous examples are:
• the MNIST dataset, where the task is to classify handwritten digits
• CIFAR-10, where the task is to classify animals and vehicles
and • CIFAR-100, where you have 100 different
classes of images The main problem is that we cannot just take
a photo and give it to the computer. We must design a way to turn that photo into numbers
in order to communicate the image to the computer. Here’s where linear algebra comes in.
Each photo has some dimensions, right? Say, this photo is 400 by 400 pixels. Each pixel
in a photo is basically a colored square. Given enough pixels and a big enough zoom-out
enough causes our brain to perceive this as an image, rather than a collection of squares.
Let’s dig into that. Here’s a simple greyscale photo. The greyscale contains 256 shades of
grey, where 0 is totally white and 255 is totally black, or vice versa.
We can actually express this photo as a matrix. If the photo is 400 by 400, then that’s
a 400 by 400 matrix. Each element of that matrix is a number from 0 to 255. It shows
the intensity of the color grey in that pixel. That’s how the computer ‘sees’ a photo.
But greyscale is boring, isn’t it? What about colored photos?
Well, so far, we had two dimensions – width and height, while the number inside corresponded
to the intensity of color. What if we want more colors?
Well, one solution mankind has come up with is the RGB scale, where RGB stands for red,
green, and blue. The idea is that any color, perceivable by the human eye can be decomposed
into some combination of red, green, and blue, where the intensity of each color is from
0 to 255 - a total of 256 shades. In order to represent a colored photo in some
linear algebraic form, we must take the example from before and add another dimension – color.
So instead of a 400 by 400 matrix, we get a 3-by-400-by-400 tensor!
This tensor contains three 400 by 400 matrices. One for each color – red, green, and blue.
And that’s how deep neural networks work with photos!
Great! Finally, dimensionality reduction.
Since we haven’t seen eigenvalues and eigenvectors yet, there is not much to say here, except
for developing some intuition. Imagine we have a dataset with 3-variables.
Visually our data may look like this. In order to represent each of those points, we have
used 3 values – one for each variable x, y, and z. Therefore, we are dealing with an
m-by-3 matrix. So, the point “i” corresponds to a vector X i, y i, and z i.
Note that those three variables: x, y, and z are the three axes of this plane.
Here’s where it becomes interesting. In some cases, we can find a plane, very close
to the data. Something like this. This plane is two-dimensional, so it is defined by two
variables, say u and v. Not all points lie on this plane, but we can approximately say
that they do. Linear algebra provides us with fast and efficient
ways to transform our initial matrix from m-by-3, where the three variables are x, y,
and z, into a new matrix, which is m-by-2, where the two variables are u and v.
In this way, instead of having 3 variables, we reduce the problem to 2 variables.
In fact, if you have 50 variables, you can reduce them to 40, or 20, or even 10.
How does that relate to the real world? Why does it make sense to do that?
Well, imagine a survey where there is a total of 50 questions. Three of them are the following:
Please rate from 1 to 5: 1) I feel comfortable around people
2) I easily make friends and
3) I like going out Now, these questions may seem different, but
in the general case, they aren’t. They all measure your level of extroversion. So, it
makes sense to combine them, right? That’s where dimensionality reduction techniques
and linear algebra come in! Very, very often we have too many variables that are not so
different, so we want to reduce the complexity of the problem by reducing the number
of variables. Thanks for watching!