When you run a linear regression analysis these are probably the most useful parameters
that you'll get out of it, you'll do a least square fit for the slope, and this case it's
9.4, and the intercept is 3.0, and I've also included 95 percent confidence intervals for
both the slope and the intercept, and if you haven't yet you might want to take a look
at the previous screencast in which I talk about, in more detail, what these confidence
intervals actually mean. What I want to talk about in this screencast is the meaning of
the p-values that you'll calculate for both the slope and the intercept. What I've generated
here are some simulated data, I've collected two simulated data points at x equals 1, x
equals 3, and five, and I fit a line to it, instead of least squared regression, and one
of the questions we want to figure out is whether or not there truly is a y-intercept
at x equals 0. Obviously with the least squares fit we would see an intercept, and I might
be in this case we're getting a fit intercept of 3. Note the question we want to answer
is "is that significant, or in reality, in whatever experiment we're running, is there
really a y-intercept, or is the fact that we're measuring in our least squares regression
we're calculating a value of b of 3, did that just occur by random chance due to the
scatter in the data." You saw how I simulated that data set, I generated a line and in that
simulated data set the true intercept is actually 0, and the slope of that line I chose a slope
of 10, and I chose an error range of plus or minus 4, is the standard deviation for
each of these normal distributions, so when I run this simulation at x equals 1, 3, and
5, I am going to randomly pick data points along this normal distribution for example,
and most of the data points we'll gather right around the mean of the distribution, and it's
possible, though it's unlikely that we'll also gather data points far away from that.
And in this screencast I'll also simulate a data set using a horizontal line, and in
this case I got a y-intercept of 30, and a slope of course of 0, and I've also got normal
distributions with a spread of plus or minus 4, a standard deviation of 4 on either side
of it. So when you calculate p-values for the slope and the intercept of a linear regression
you're actually performing two hypothesis tests. The first null hypothesis is that the
intercept is actually equal to 0, and its alternative hypothesis is that the intercept
is non-zero, and similarly the second hypothesis test, the null hypothesis is that the slope
is 0, and the alternative hypothesis is that the slope is non-zero. So in these two graphs
I am showing two cases in which the null hypothesis for the intercept is true, and in the second
case I am showing a case in which the null hypothesis for the slope is true. So in this
case on the left the y-intercept is indeed 0, and the graph on the right, the slope is
indeed 0. But in both cases we don't ever get to know what the true line is, we are
just trying to approximate it when we measure our data points, so we're trying to get as
close as possible, and we're trying to figure out what these lines actually look like. So
when we randomly collect data with normal distributions, it's quite possible, although
it might be unlikely, that we could collect data sets in which the points were randomly
selected relatively high to that true line, and what you would find when you did a linear
regression on that we would see a case in which the y-intercept was a great deal above
0, and we might falsely conclude that the null hypothesis is false, and if that's the
case we've made a type I error. Similarly for the case on the right, when the null hypothesis
is true for the slope, the slope is actually 0, it's quite possible, although maybe a little
bit unlikely, that we would collect data points that look something like this, and then when
you fit a linear regression to that the line would look something like this, and we would
look at that and we'd say, well the slope isn't 0, it's something negative, and in that
case we've made at type I error for the slope. And the p-value that you generate in both
cases gives you an indication, of the odds that you made a type I error, and a rule of thumb
is that the smaller the p-value for the intercept the more convinced you can be that the intercept
is non-zero, or we could also say the more convinced you could be that it's significant,
and similarly the smaller the p-value for the slope the more convinced you could be
that the slope is something other than zero. And as usual you'd typically say that if the
p-value is less than 0.05, you would say that it is significant, meaning we have enough
evidence to argue that the slope, or the intercept are non-zero, and in this screencast you and
I both know that the null hypothesis for the graph on the left, the null hypothesis for
the intercept is true, and you and I both know that the null hypothesis, the graph on
the right, for the slope is true. But in reality we don't ever get to know that so the best
we can do is make some statistical arguments, and just saying the odds that we are correct.
And just so you can see it, here is a third case in which the null hypothesis for the slope
is true, and the null hypothesis for the intercept is true, and what you'd, what you would observe
is a horizontal line at y equals 0. So when you measure data for something like this odds
are you'll find that the p-value for both the slope and the intercept are fairly large,
or we'd say both the slope and the intercept are insignificant. Here's the original data
set I showed you at the beginning, and I am also showing you how I came up with that data
set, or how I simulated it. Again I've got the true line, the true behavior of the experiment
shown here in the thick line, and the dashed line shows the regression analysis that you
would get just by performing a least squares regression, and I am also showing a p, or
the p-value for the intercept, and the p-value for the intercept is a great deal larger than
0.05, and it doesn't give us enough evidence to refute the null hypothesis, so it doesn't
give us enough evidence to say that the intercept is something other than zero. So maybe a dirty
way to explain this would be, well we've got a little bit of wiggle room to fit the data,
you could imagine well both of these look pretty good for the fit of the data, and they
tend to bracket zero, it might be another way to think about this. And also note that
the p-value for the intercept is larger than 0.05, and consequently the 95 percent confidence
interval for the intercept will bracket zero, so the 95 percent confidence interval for
b, the low end of that would be less than 0, and the high end is greater than 0 because
it contains 0, it's another way to say that the y-intercept is insignificant. Look at
the p-value for the slope though, it's a tiny value, 1 times 10 to the negative 4th, and
what that says is that it's highly unlikely that the null hypothesis for the slope is
true. It's highly unlikely that the true slope is actually equal to zero, and we measure
this trend by chance, so in this case we can clearly reject the null hypothesis that the
slope is 0, and of course we would fit a line that in this case had a positive slope, and
in addition the 95 percent confidence interval for the slope doesn't come anywhere near 0,
so we get a confidence interval that's far away from zero, and consequently a very small
p-value. So here is another example in which I set the null hypothesis for the slope, I've
made it true, so I've set the actual slope, the thick black line here, equal to zero,
and I generated some random data, the graph on the right is the graph that you would actually
see, and the question is whether or not these data suggest that there is any sort of slope
in the true experimental analysis, so you run your regression analysis, and the first
thing that you would observe is that the p-value for the slope is greater than 0.05, and it
suggests that the true slope is not any different than 0, and again we look at the 95 percent
confidence interval for the slope, and that confidence interval does contain zero, and
it also suggests that the slope is insignificant. However, the intercept we get a very small
p-value, suggesting that there is no way that the intercept is actually equal to zero, it's
highly unlikely that that would have occurred in the 95 percent confidence interval for
the intercept doesn't come anywhere close to zero. So to simulate this imagine that
you and 99 of your friends ran the exact same experiment and collected data. What I am showing
is a case in which the null hypothesis for the intercept is true, so the true intercept
is actually equal to zero, and if we get a p-value that is 0.05 or less, then we made
a type I error, we've incorrectly concluded that the intercept is non-zero, so I'll cycle
through, and on run number 4 we made a type I error, so in this case look at the p-value
for the intercept, it's 0.02, and the graph on the right, it makes it look incorrectly
that the intercept is some negative value, or something fairly clearly other than zero,
and you just look at the data themselves, and that's all you get to see in real life,
you don't ever get to know that you made a type I error, but in the graph on the left
becomes maybe a little bit more obvious that the intercept ought to equal 0. And I've just
been cycling through. And on run number 30 again we made a type 1 error, where the p-value
for the intercept is less than 0.05, so I'll keep going through, and remember we're looking
for about 5 type I errors when I make 100 runs, it could be a little more, a little less,
but on average we'll make 5 type I errors. And as it turns out for those 100 runs, ended
up making six type I errors. So that means six times we concluded that there indeed was
a non-zero y-intercept when in fact there wasn't, and again I can do the same thing
for the slope, here I set up the null hypothesis being true for the slope, the slope is indeed
zero, so every time we get a p-value for the slope that's less than 0.05, we've made a
type I error for the slope. So I'll cycle through like I did before, and here
is a case in which we made a type I error for the slope. The p-value for the slope is
much less than 0.05, and when you look at the data on the right that's what you would
see in real life, and when you fit a regression to that you would say clearly there's a slope,
but as it turns out it was all due to chance, it was all due to random noise in your experiment,
and you made a type I error but you didn't even know it. So I'll just continue to cycle
through. Here's another case in which we made a type I error. Remember we're looking, probably
be about 5 out of the 100 runs, another case for a type I error. Here's another one. Here's
another one on run 41. And here's another case, and typically if you see data points that
are clustered really tightly around a line you got higher evidence for a slope, but you
could still be making a type I error. So actually I already have 8 of these, we got 9 type 1
errors on this particular one. So remember that this is all on average, up to 10 now.
So in this case we've actually made 11 type I errors over 100 runs. So it's not, I mean
it's law of chance, but again on average we would see about 5 for every 100 runs. So I'll
rerun this with 1000 runs, and we'll expect something close to 50, and the more runs you
carry out the closer you will get to the true average. So here I've rerun the simulation
1000 times to show you that on average we'll get something close to 5 percent of those
runs will be a type I run, so in this case we're close to about 50 out of 1000 runs we
made a type I error, but 950 times the hypothesis was correct.