We have introduced the concept of expected value or mean,
which tells us the average value of a random variable.
We will now introduce another quantity, the variance, which
quantifies the spread of the
distribution of a random variable.
So consider a random variable with a given PMF, for example
like the PMF shown in this diagram.
And consider another random variable that happens to have
the same mean, but it's distribution
is more spread out.
So both random variables have the same mean, which we denote
by mu, and which in this picture would be somewhere
around here.
However, the second PMF, the blue PMF, has typical outcomes
that tend to have a larger distance from the mean.
By distance from the mean what we mean is that if the result
of the random variable, its numerical value, happens to
be, let's say for example, this one, then this quantity
here is X minus mu is the distance from the mean, how
far away the outcome of the random variable happens to be
from the mean of that random variable.
Of course, the distance from the mean is a random quantity.
It is a random variable.
Its value is determined once we know the outcome of the
experiment and the value of the random variable.
What can we say about the distance from the mean.
Let us calculate its average or expected value.
The expected value of the distance from the mean, which
is this quantity, using the linearity of expectations, is
equal to the expected value of X minus the constant mu.
But the expected value is by definition equal to mu.
And so we obtain zero.
So we see that the average value of the distance from the
mean is always zero.
And so it is uninformative.
What we really want is the average absolute value of the
distance from the mean, or something with this flavor.
Mathematically, it turns out that the average of the
squared distance from the mean is a better behaved
mathematical object.
And this is the quantity that we will consider.
It has a name.
It is called the variance.
And it is defined as the expected value of the squared
distance from the mean.
The first thing to note is that the variance is always
non-negative.
This is because it is the expected value of non-negative
quantities.
How exactly do we computer the variance?
The squared distance from the mean is really a function of
the random variable X. So it is a function of the form g of
X, where g is a particular function defined this way.
So we can use the expected value rule applied to this
particular function g.
And we obtain the following.
So what we have to do is to go over all numerical values of
the random variable X. For each one, calculate its
squared distance from the mean and weigh that quantity
according to the corresponding probability of that particular
numerical value.
One final comment, the variance is a bit hard to
interpret, because it is in the wrong units.
If capital X corresponds to meters, then the variance has
units of meters squared.
A more intuitive quantity is the square root of the
variance, which is called the standard deviation.
It has the same units as the random variable and captures
the width of the distribution.
Let us now take a quick look at some of the properties of
the variance.
We know that expectations have a linearity property.
Is this the case for the variance as well?
Not quite.
Instead we have this relation for the variance of a linear
function of a random variable.
Let us see why it is true.
We use the shorthand notation mu for the expected value of
X. We will proceed one step at a time and first consider what
happens to the variance if we add the
constant to a random variable.
So let Y be X plus some constant b.
And let us just define nu to be the expected value of Y,
which, using linearity of expectations, is the expected
value of X plus b.
Let us now calculate the variance.
By definition the variance of Y is the expected value of the
distance squared of Y from its mean.
Now we substitute, because in this case Y is
equal to X plus b.
Whereas the mean, nu, is mu plus b.
And now we notice that this b cancels with that b.
And we are left with the expected value of X minus mu
squared, which is just the variance of X. So this proves
this relation for the case where a is equal to 1.
The variance of X plus b is equal to the variance of X. So
we see that when we add a constant to a random variable,
the variance remains unchanged.
Intuitively, adding a constant just moves the entire PMF
right or left by some amount, but without
changing its shape.
And so the spread of this PMF remains unchanged.
Let us now see what happens if we multiply a random variable
by a constant.
Let again nu be the expected value of Y. And so in this
case by linearity this is equal to a times the expected
value of X. So it is a times mu.
We calculate the variance once more using the definition and
substituting in the place of Y what Y is in this case--
it's aX--
and subtracting the mean of Y, which is a mu, squared.
We take out a factor of a squared.
And then we use linearity of expectations to note that this
is a squared times the expected value of X minus mu
squared, which is a squared times the variance of X.
So this establishes this formula for the case where b
equals zero.
Putting together these two facts, if we multiply a random
variable by a, the variance gets multiplied by a squared.
And if we add a constant, the variance doesn't change.
And this establishes this particular fact.
As an example, the variance of, let's say, 3 minus 4X is
going to be equal minus 4 squared times the variance of
X, which is 16 times the variance of X.
Finally, let me mention an alternative way of computing
variances, which is often a bit quicker.
We have this useful formula here.
We will see later a few examples of how it is used,
but for now let me just show why it is true.
We have by definition that the variance of X is the expected
value of X minus mu squared.
Now let us rewrite what is inside the expectation by just
expanding this square, which is [X squared minus]
2 mu X plus mu squared.
Using linearity of expectations, this is broken
down into expected value of X squared minus the expected
value of two times mu X. But mu is a constant.
So we can take it outside the expected value.
And we're left with 2mu expected value
of X plus mu squared.
But remember that mu is just the same as the expected value
of X. So what we have here is twice the expected value of X,
squared, plus the expected value of X, squared, and that
leaves us just minus the expected value of X, squared.
So we will now move in the next segment into a few
examples of variance calculations.