Welcome to a lesson on performing
least squares linear regression using matrices.
If we're given points as labeled here,
the last squares regression line can be given by:
f of x equals b plus mx,
where b would be the y-intercept,
and m would be the slope of the line.
And this will minimize the sum of the squared errors,
which are the errors in using the regression function
f of x to estimate the true y values.
So e sub i would be the error, or the difference between
the true y value and the function value from the line.
And if we square all these errors,
we have the sum of the squared errors.
Let's see if we can set this up
as a system of equations,
and then solve using matrices.
Every given point would create a linear equation
in the form of y sub n, the given y value,
must equal b plus m times x sub n,
plus e sub n,
where again, b is the y-intercept, m is the slope,
x is the given x value, and then this e here on the end
is the small error that we would have
by using the regression line to make the prediction
for the true y value.
So again, each point creates a linear equation,
and together we have a system of equations
which we can write as a matrix equation.
To do this, we start with forming matrix Y,
which contains the given y values.
Matrix X will have two columns:
a column of ones, and then a column of the given x values.
Matrix A will contain the y-intercept and slope.
And matrix E will contain all of the errors.
This gives us the matrix equation
Y equals X times A plus E.
Our goal is to solve this equation for A.
But before we do this,
let's see why these ones here are necessary.
If we were to start multiplying matrix X and matrix A,
row one times column one.
One times b gives us b, our y-intercept,
and x sub one times m
gives us the slope times the variable x.
So this one is what gives us the y-intercept
in each of these linear equations.
Now, we're not going to derive these formulas here,
but to solve this matrix equation,
A, remember that gives us the y-intercept and slope,
is equal to this product here.
Where we have the inverse of the product of
the transpose of matrix X and matrix X,
times the product of the transpose of matrix X
and matrix Y.
And the sum of the squared errors is equal to
the transpose of matrix E times matrix E.
Let's take a look at an example.
We want to determine the least squares regression line
from the given data, where x is the price in dollars,
and y is the demand in monthly sales.
We can see from the graph of the data,
the linear regression line is going to be a pretty good fit
for this data.
Let's start by setting up our matrices.
We want to start by setting up matrix Y and matrix X.
Matrix Y is the column matrix, with the given Y values.
So it's a five by one matrix.
Matrix X will be a five by two matrix,
because remember the first column will consist of ones,
second column will consist of the given x values.
So given these two matrices, we have the matrix equation
Y equals A times X plus E.
We won't be able to find matrix E
until after we solve for matrix A,
and then determine the errors using the regression line.
So now we're going to find this product here,
but we're going to do it in pieces.
We're going to start by finding the product of
the transpose of matrix X and matrix X.
Well here's matrix X.
Notice how again it's five by two matrix.
So the transpose is going to be a two by five matrix.
So this first column becomes row one,
the second column becomes row two.
Notice matrix X is a five by two matrix.
The fives tell us that we can perform the multiplication,
the twos tell us the dimension of our product.
It's a two by two matrix,
which I've already found to save some time.
Now, for the next step,
we'll find the inverse of this product.
Because this product is a two by two matrix,
remember we can take the shortcut
where we use this formula here
to find the inverse of our matrix.
Which, again, I've already done here.
So for the next step we want to find
the product of the transpose of matrix X and matrix Y.
We already found the transpose of matrix X earlier.
Here is matrix Y.
Again, notice how this is a two by five matrix.
This is a five by one matrix.
So the product is going to be a two by one matrix,
which we see here.
Again, I've already found this product due to time.
Now that we have all the pieces,
we can finally find matrix A,
which, again, will give us the y-intercept and slope
of our regression line.
So, matrix A is equal to the inverse of this product,
which we have here,
times this product,
which we have here.
This gives us a two by one matrix,
where the elements are approximately
211, and negative 1.7.
So 211 is b, or the y-intercept of our regression line,
and -1.7 is the slope of our regression line.
So we have f of x equals 211 minus 1.7x.
If we want to put this in sloped intercept form,
we could say f of x equals -1.7x,
plus 211.
Here's a graph of our line over the data.
You can see that the line does to the data pretty well.
So it is a good model for the data.
Now, we're also asked to find
the sum of the squares of the errors.
So to do this, I recommend creating a table similar to this,
where the first column is the given x values,
the second column is the given y values.
This third column, here, are the y values
or function values from the regression equation,
f of x equals 211 minus 1.7x.
So the difference between y sub i and f of x sub i
is the error which we see here in the third column.
Which we need to form matrix E,
to find the sum of the squares of the errors.
So to find the sum of the squares of the errors,
we have the transpose of matrix E times matrix E.
Notice how matrix E is a column matrix,
the transpose is a row matrix,
and this product is equal to 207.65,
which is the sum of the square of the errors.
Let's finish by taking a look at a question.
Using our regression function or regression line,
we want to know what the meaning of the slope is,
and also we want to predict the demand
if the price is set at $54.
Well, the slope is the ratio of the change of y
to the change of x.
In this case, y is demand.
So our slope represents the ratio of change of demand,
to change of x, which is change in price.
Next, our slope is negative 1.7,
which as a ratio would be 1.7 over one.
So the positive one represents
an increase in price of $1,
and the negative 1.7 represents a decrease
of 1.7 in demand.
So, to put this in a sentence,
according to the model, if the price increases by $1,
we can expect
a decrease in demand
of 1.7 units per month.
Next, we're asked to predict the demand
if the price is set at $54.
Well, x represents the price in dollars,
so for the second part, we're going to determine
the function value at x equals 54.
f of 54
is equal to 211
minus 1.7 times 54,
what is equal to 211
minus
91.8,
which would be 119.2.
So, according to the model,
if the price is $54,
we can expect a monthly demand of
approximately 119 units.
Okay, I think we'll go ahead and stop here.
I hope you found this helpful.