let's talk a little bit about analysis of variance as a concept and we'll also
see how it's applied to the one-way analysis of variance test. So as noted,
the concept of analysis of variance flows through a lot of statistical
methods; so we want to get a good conceptual understanding of exactly what
does it mean to analyze variance and what do we mean by sums of squares,
explained or unexplained sums of squares and so on. So we're gonna work with a
simplified example here; so we're gonna continue on with that example of
comparing weight loss for diets but here I've simplified it to only looking at
three diets and we're going to compare the weight loss of three observations in
each group and again this is done so that we can have a simple set of data
and the pictures we're going to draw don't get over complicated with having
you know 60 different points on there; so here we can see for diet A, here's the weight loss
for the three individuals for diet B here's the weight loss for the
three individuals and for diet C again the weight loss for the three
individuals and what I've drawn in here sometimes gets called the overall mean
or the grand mean: it's the average weight loss in the study of all these
individuals ignoring the fact that they might be on different diets. So the first
thing we're going to do is take a look at the idea of total sum of squares so
we can look at how far does each individual move from the overall mean
okay so I'm going to draw those in here so here we're looking at how far is each
person from the overall mean. These are giving us some idea of the total
variability in weight loss, so what we call the sum of squares, so we write
SS for sum of squares total, total sum of squares is we're going to sum over all
of these observations, all nine here, how far is the individual, how far is
each point from the overall mean, okay I am just going to write
overall y-bar meaning the overall mean of weight loss in all the study squared.
So again if you want to think of it in terms of the graph we're going to
sum over all of these distances here squared
how far is each individual from the overall mean, square that do that for all
of them add them all up: total sum of squares; this we can use to get to the
idea of the total variability in Y or total variability in weight loss so here
s squared the variance, the sample variance for the total or
for Y is rather than making it too messy I'm just going to write this here the
sum of squares total so this top here divided by its degrees of freedom
n minus 1(n-1); what I want to point out here is this is getting us the total variance
in Y : how far is each individual from the overall mean squared divided by its
degrees of freedom okay total variability in Y or total variability in
weight loss so the top half of the formula for variance is what gets called
a sum of squares, we're summing up a bunch of squares so we're going to
analyze the total variability. to do that we're going to try and take the
total variability and weight loss and separate it into a few different parts
so let's start to build up that concept: so the first thing I want to think about
is why did we not see everyone having the exact same weight loss and how
that's right, WL to shorten it, so some people lost a lot of weight some people
lost relatively less and you might seem like a silly question but let's think
about it for a moment why did not everyone lose exactly say 10 pounds and
that's a few reasons the first one what you might be thinking is well diets
might be different so diets may differ from each other which is true
right maybe this person lost a lot more than this person because diet A might be
more effective than diet B, well what about all these people who are all on
diet A, how come they didn't all lose the exact same amount of weight?
Of course we wouldn't expect that in a real world but we're trying to think our way
through the concepts. So I'm gonna say that people are different there's
biological variability take two people put them on the exact same diet they're
not going to lose the exact same amount of weight words that we attach to these
weight loss might be different because diets are different we call that
explained by diet they're explained by X then the fact that people are different
this is just random or unexplainable so we call it unexplained by diet or by X
so again to repeat that everyone in this study had slightly different weight loss
and that's for a few reasons why did this person here not lose the same as
this person here well maybe diet A and diet B are
different taking two people who are on the same diet why did they not lose the
exact same amount of weight and we can't explain why that's just random
variability may be biological differences it's due to other factors
other than the diet right since they're both on the same diet it's not
attributable to diet so what we'd like to do is take this idea of total
variability so I'm gonna write that here the sum squares total and we're going
to see how this can be divided into two parts okay we can take it and we can
separate it into sum of squares that we're gonna say is explained by diet
and a sum squares that's unexplained and we're going to go through
graphically you can take a look at the algebra of of how the total sum of
squares can be separate into these two parts again if you have a stronger
background in mathematics you can work your way through it and have a good
understanding of it and you can take a look at that it's in a lot of sets of
notes and textbooks I'm going to get a graphical version of that okay before I
do that I just want to attach a few other words here sometimes the
variability that is explained by diets being different gets called Between (group variability)
and the unexplained sometimes gets called Within (group variability). so I'm going to use these
I'm going to alternate between calling it between and within or explained and
unexplained okay we'll see it graphically in a few different ways; okay
let's start to build up that idea. So what I'm gonna do, again we are looking
at these concept graphically I'm going to pick on one observation so I'll pick
on this one here and we're gonna try and get at the concept of how we can take
the total sum of squares and separate it into how much do we think is explained
by diet, how much is not explained by diet. This picture here shows the total
variability and now we'd say well we shouldn't look at how far each person is
from the overall mean, diets might be different, someone who's on diet A
let's say this person here they shouldn't be losing the average weight
loss in the study, you know statistically we'd expect them to be on the average
for diet A, so here I've drawn in the average of those three, the sample
mean or sample average for those three individuals so I'd say this person here
I wouldn't expect them to be the average weight loss in the study, diet A appears
to be a bit better I'd expect them to be the average weight
loss for diet A. so this jump up here we're going to call the explained
out of this total variability, I can explain this part I wouldn't expect them
to be sitting on the mean in the overall study I'd expect to be sitting on the
mean for diet A, right, they're part of diet A, but they jumped a bit above the
means okay and why do they jump above the mean I can't explain that
it's random variability or it's due to other factors okay but other factors
that are not diet so this is unexplained again unexplained by diet let's write
some of these out here so again as I mentioned you can take a look at the the
algebra of this but the total sum of squares can be separated into these two
parts we have the sum of squares between
again if you want you can think of it as the explained it's part that's explained
by diet or explained by X variable to get this we're gonna sum again over all
of the observations how far is the group's mean so I'm going to write it
this way the group's mean from the overall mean squared right so again
looking there how far is the group's mean from the overall mean right this
distance here and square that those are going to contribute to the sum of
squares between okay we're gonna do that for all the observations here in the
picture we're just highlighting one to focus on the concept but the sum of
squares between is for all these observations how far is the group mean
from the overall mean squared again this contributes to what we can call the
variance right the sample variance between groups or sometimes it gets
called the Mean Squared Between groups is the sum of squares between groups
divided by its degrees of freedom and which happen to be K minus 1 (k-1) okay again
so k being the number of groups 1 to K equals 3 groups minus 1 we've got the
mean for group 1 for group 2 and group 3 right those are our three
degrees of freedom to start with and we lose one having to estimate the overall mean
so maybe to make this complete I should add in the mean for Group B say the mean
for group C now we've gotten the unexplained portion we've got the sum of
squares within a group we can also call this the unexplained the part of the
variability that's not explained by the diet and here again we're going to sum
over all of the observations how far is the individual from their group specific
mean, so how far is the individual from their group's mean and squared so
mathematically if we take this total distance all right the total sum of
squares so this distance squared we can separate it into being this distance
squared plus this distance squared you'll notice mathematically if you work
it out this squared is not equal to this squared plus this squared but the
concept holds over all the observations if for every single observation we take
all these totals the blue distances squared that's going to be equal to the
sum of squares explained plus the sum of squares unexplained let me write it down
here the sum of squares between plus the sum of squares within, okay and again
this is the top half of the variance formula if we want the variance or
variability within a group sometimes called the Mean Squared Within it's the
sum of squares within divided by this degrees of freedom which are n minus K
and again calculating within group variability that's going to come from
squaring the n observations and we lose the three
degrees of freedom we have to estimate each of the group meets so K being three
you start with n total observations of nine and we lose three degrees of
freedom we've had to estimate the three group needs and calculating that so I
just want to redraw this picture a little bit and add one more visual I'm
going to draw the same label the parts slightly differently that's going to
help connect a bit better rather than thinking it explained and unexplained
the between and the within. So we've already drawn a smaller version of this
plot here again just to highlight the between and within group variability so
we've taken this total variability how far is each individual from the mean
and we saw that this can be separated to that that is explained by diet or going
on between diets and that which cannot be explained by diet or was within
a diet. So first let's highlight the between we
saw that's looking at how far is the group mean from the overall mean squared
so that's looking at this distance here how far is the group mean from the
overall mean and squaring those okay so that's the part that contributes to be
between group variability how far is each group's mean from the overall mean
squared, sum those up for all observations; within group variability is
looking at how far are the individuals from their groups specifically and how
far is each individual from their groups mean
so again this is looking at variability within a group or within a diet so
mathematically that total sum of squares can be separated into two parts how much
of it is going on between diets and how much of it goes on within a diet so it's
important to note there's many names that get thrown around for these so it
can be helpful for you when you look at different sources to try and match the
different names that gets used often the sum of squares between groups gets called
the explained sum of squares or sometimes the sum of squares model
Sum of squares treatment, sum of squares regression okay
similar to this here's the mean squares between, the explained mean squared, the
mean squared model treatment, regression these are terms that usually get used
the sum of squares within a group sometimes gets called sum of squares
that is unexplained, sum squared error or some squared residual, okay so again lots
of different names that get used for the exact same concept. A few kind of
important notes to close on: this picture is really good I think getting
at the concept, there is a slight flaw in it and that here we're looking at the
distances how far is an observation from the mean and these sums of squares are
actually those units squared okay so just to pick on one observation picking
on this one here we'd actually say how far is the group mean from the overall
mean this is the explained for that individual this is the unexplained for
that individual okay so I said the picture doesn't fully show stuff there
are some slight flaws in it and the flaw is that these are in the distances the
math is actually the distance is squared but when we do it over the collective
the blue distances squared add up to the orange distances squared plus the pink
distances squared okay so it's just worth pointing that out I guess a few
other terms that I want to point out before we we close on this, is that you
can think of these in some sense it's taking the total sum of squares and
we're separating them into signal and the noise
This is how much is explained or due to treatment the signal we're getting, this is how much is
unexplainable or or noise okay so again we can think of this here and it's
giving us something about the signal and this here the noise. so when working
through and building up the test statistic for one-way analysis of
variance (ANOVA) what we want to do is we're going to end up wanting to compare these
two the variability between groups - the
variability within groups and so we're going to start to do that essentially
we're going to take the ratio of these two but let's get into building up the
test statistic for one-way analysis of variance. thanks for watching our video
subscribe to marinstatslectures