Our discussion of random variable so far has involved
nothing but standard probability calculations.
Other than using the PMF notation, we have
done nothing new.
It is now time to introduce a truly new concept that plays a
central role in probability theory.
This is the concept of the expected value or expectation
or mean of a random variable.
It is a single number that provides some kind of summary
of a random variable by telling us what
it is on the average.
Let us motivate with an example.
You play a game of chance over and over, let
us say 1,000 times.
Each time that you play, you win an amount of money, which
is a random variable, and that random variable takes the
value 1, with probability 2/10, the value of 2, with
probability 5/10, the value of 4, with probability 3/10.
You can plot the PMF of this random variable.
It takes values 1, 2, and 4.
And the associated probabilities are
2/10, 5/10, and 3/10.
How much do you expect to have at the end of the day?
Well, if you interpret probabilities as frequencies,
in a thousand plays, you expect to have about 200 times
this outcome to occur and this outcome about 500 times and
this outcome about 300 times.
So your average gain is expected to be your total
gain, which is 1, 200 times, plus 2, 500
times, plus 4, 300 times.
This is your total gain.
And to get to the average gain, you divide by 1,000.
And the expression that you get can also be written in a
simpler form as 1 times 2/10 plus 2 times 5/10
plus 4 times 3/10.
So this is what you expect to get, on the average, if you
keep playing that game.
What have we done?
We have calculated a certain quantity which is a sort of
average of the random variable of interest.
And what we did in this summation here, we took each
one of the possible values of the random variable.
Each possible value corresponds to
one term in the summation.
And what we're adding is the numerical value of the random
variable times the probability that this
particular value is obtained.
So when x is equal to 1, we get 1 here and then the
probability of 1.
When we add the term corresponding to x equals 2,
we get little x equals to 2 and next to it the probability
that x is equal to 2, and so on.
So this is what we call the expected value of the random
variable x.
This is the formula that defines it, but it's also
important to always keep in mind the interpretation of
that formula.
The expected value of a random variable is to be interpreted
as the average that you expect to see in a large number of
independent repetitions of the experiment.
One small technical caveat, if we're dealing with a random
variable that takes values in a discrete but infinite set,
this sum here is going to be an infinite sum
or an infinite series.
And there's always a question whether an infinite series has
a well-defined limit or not.
In order for it to have a well-defined limit, we will be
making the assumption that this infinite series is, as
it's called, absolutely convergent, namely that if we
replace the x's by their absolute values--
so we're adding positive numbers,
or nonnegative numbers--
the sum of those numbers is going to be finite.
So this is a technical condition that we need in
order to make sure that this expected value is a
well-defined and finite quantity.
Let us now calculate the expected value of a very
simple random variable, the Bernoulli random variable that
takes the value 1 with probability p and the value 0
with probability 1 minus p.
The expected value consists of two terms.
X can take the value 1.
This happens with probability p.
Or it can take the value of zero.
This happens with probability 1 minus p.
And therefore, the expected value is just equal to p.
As a special case, we may consider the situation where X
is the indicator random variable of a certain event,
A, so that X is equal to 1 if and only if event A occurs.
In this case, the probability that X equals to 1, which is
our parameter p, is the same as the probability
that event A occurs.
And we have this relation.
And so with this correspondence, we readily
conclude that the expected value of an indicator random
variable is equal to the probability of that event.
Let us move now to the calculation of the expected
value of a uniform random variable.
Let us consider, to keep things simple, a random
variable which is uniform on the set from 0 to n.
It's uniform, so the probability of the values that
it can take are all equal to each other.
It can take one of n plus 1 possible values, and so the
probability of each one of the values is 1 over n plus 1.
We want to calculate the expected value
of this random variable.
How do we proceed?
We just recall the definition of the expectation.
It's a sum where we add over all of the possible values.
And for each one of the values, we multiply by its
corresponding probability.
So we obtain a summation of this form.
We can factor out a factor of 1 over n plus 1, and we're
left with 0 plus 1 plus all the way up to n.
And perhaps you remember the formula for us summing those
numbers, and it is n times n plus 1 over 2.
And after doing the cancellations, we obtain a
final answer, which is n over 2.
Incidentally, notice that n over 2 is just the midpoint of
this picture that we have here in this diagram.
This is always the case.
Whenever we have a PMF which is symmetric around a certain
point, then the expected value will be
the center of symmetry.
More general, if you do not have symmetry, the expected
value turns out to be the center of gravity of the PMF.
If you think of these bars as having weight, where the
weight is proportional to their height, the center of
gravity is the point at which you should put your finger if
you want to balance that diagram so that it doesn't
fall in one direction or the other.
And we now close this segment by providing one more
interpretation of expectations.
Suppose that we have a class consisting of n students and
that the ith student has a weight which
is some number xi.
We have a probabilistic experiment where we pick one
of the students at random, and each student is equally likely
to be picked as any other student.
And we're interested in the random variable X, which is
the weight of the student that was selected.
To keep things simple, we will assume that the
xi's are all distinct.
And we first find the PMF of this random variable.
Any particular xi that this possible is associated to
exactly one student, because we assumed that
the xi's are distinct.
So this probability would be the probability or selecting
the ith student, and that probability is 1 over n.
And now we can proceed and calculate the expected value
of the random variable X. This random variable X takes
values, and the values that it takes are the xi's.
A particular xi would be associated with a probability
1 over n, and we're adding over all the i's or over all
of the students.
And so this is the expected value.
What we have here is just the average of the weights of the
students in this class.
So the expected value in this particular experiment can be
interpreted as the true average over the entire
population of the students.
Of course, here we're talking about two
different kinds of averages.
In some sense, we're thinking of expected values as the
average in a large number of repetitions of experiments.
But here we have another interpretation as the average
over a particular population.