We have seen that several properties, such as, for
example, linearity of expectations, are common for
discrete and continuous random variables.
For this reason, it would be nice to have a way of talking
about the distribution of all kinds of random variables
without having to keep making a distinction between the
different types--
discrete or continuous.
This leads us to describe the distribution of a random
variable in a new way, in terms of a so-called
cumulative distribution function or CDF for short.
A CDF is defined as follows.
The CDF is a function of a single argument, which we
denote by little x in this case.
And it gives us the probability that the random
variable takes a value less than or equal to this
particular little x.
We will always use uppercase Fs to indicate CDFs.
And we will always have some subscripts that indicate which
random variable we're talking about.
The beauty of the CDF is that it just involves a
probability--
a concept that is well defined, no matter what kind
of random variable we're dealing with.
So in particular, if X is a continuous random variable,
the probability that X is less than or equal
to a certain number--
this is just the integral of the PDF over that range from
minus infinity up to that number.
As a more concrete example, let us consider a uniform
random variable that ranges between a and b, and let us
just try to plot the corresponding CDF.
The CDF is a function of little x.
And the form that it takes depends on what kind of x
we're talking about.
If little x falls somewhere here to the left of a, and we
ask for the probability that our random variable takes
values in this interval, then this probability will be 0
because all of the probability of this uniform is
between a and b.
Therefore, the CDF is going to be 0 for values of x less than
or equal to a.
How about the case where x lies somewhere
between a and b?
In that case, the probability that our random variable falls
to the left of here--
this is whatever mass there is under the PDF when we consider
the integral up to this particular point.
So we're looking at the area under the PDF up to this
particular point x.
This area is of the form the base of the rectangle, which
is x minus a, times the height of the rectangle, which is 1
over b minus a.
This is a linear function in x that takes the value of 0 when
x is equal to a, grows linearly, and when x reaches a
value of b, it becomes equal to 1.
How about the case where x lies to the right of b?
We're talking about the probability that our random
variable takes values less than or equal to this
particular x.
But this includes the entire probability
mass of this uniform.
We have unit mass on this particular interval, so the
probability of falling to the left of here is equal to 1.
And this is the shape of the CDF for the case of a uniform
random variable.
It starts at 0, eventually it rises, and eventually it
reaches a value of 1 and stays constant.
Coming back to the general case, CDFs are very useful,
because once we know the CDF of a random variable, we have
enough information to calculate anything we might
want to calculate.
For example, consider the following calculation.
Let us look at the range of numbers from minus infinity to
3 and then up to 4.
If we want to calculate the probability that X is less
than or equal to 4, we can break it down as the
probability that X is less than or equal to 3--
this is one term--
plus the probability that X falls between 3 and 4, which
would be this event here.
So this equality is true because of the additivity
property of probabilities.
This event is broken down into two possible events.
Either x is less than or equal to 3 or x is bigger than 3 but
less than or equal to 4.
But now we recognize that if we know the CDF of the random
variable, then we know this quantity.
We also know this quantity, and this allows us to
calculate this quantity.
So we can calculate the probability of a
more general interval.
So in general, the CDF contains all available
probabilistic information about a random variable.
It is just a different way of describing the probability
distribution.
From the CDF, we can recover any quantity we
might wish to know.
And for continuous random variables, the CDF actually
has enough information for us to be able to recover the PDF.
How can we do that?
Let's look at this relation here, and let's take
derivatives of both sides.
On the left, we obtain the derivative of the CDF.
And let's evaluate it at a particular point x.
What do we get on the right?
By basic calculus results, the derivative of an integral,
with respect to the upper limit of the integration, is
just the integrand itself.
So it is the density itself.
So this is a very useful formula, which tells us that
once we have the CDF, we can calculate the PDF.
And conversely, if we have the PDF, we can find the CDF by
integrating.
Of course, this formula can only be correct at those
places where the CDF has a derivative.
For example, at this corner here, the derivative of the
CDF is not well defined.
We would get a different value if we differentiate from the
left, a different value when we differentiate from the
right, so we cannot apply this formula.
But at those places where the CDF is differentiable, at
those places we can find the corresponding
value of the PDF.
For instance, in this diagram, at this point the CDF is
differentiable.
The derivative is equal to the slope, which is this quantity.
And this quantity happens to be exactly the same as the
value of the PDF.
So indeed, here, we see that the PDF can be found by taking
the derivative of the CDF.
Now, as we discussed earlier, CDFs are relevant to all types
of random variables.
So in particular, they are also relevant to discrete
random variables.
For a discrete random variable, the CDF is, of
course, defined the same way, except that we calculate this
probability by adding the probabilities of all possible
values of the random variable that are less than
[or equal to]
the particular little x that we're considering.
So we have a summation instead of an integral.
Let us look at an example.
This is an example of a discrete random variable
described by a PMF.
And let us try to calculate the corresponding CDF.
The probability of falling to the left of this number, for
example, is equal to 0.
And all the way up to 1, there is 0 probability of getting a
value for the random variable less than that.
But now, if we let x to be equal to 1, then we're talking
about the probability that the random variable takes a value
less than or equal to 1.
And because this includes the value of 1, this probability
would be equal to 1/4.
This means that once we reach this point, the value of the
CDF becomes 1/4.
At this point, the CDF makes a jump.
At 1, the value of the CDF is equal to 1/4.
Just before 1, the value of the CDF was equal to 0.
Now what's the probability of falling to the left
of, let's say, 2?
This probability is again 1/4.
There's no change in the probability as we keep moving
inside this interval.
So the CDF stays constant, until at some point we reach
the value of 3.
And at that point, the probability that the random
variable takes a value less than or equal to 3 is going to
be the probability of a 3 plus the probability of a 1 which
becomes 3 over 4.
For any other x in this interval, the probability that
the random variable takes a value less than this number
will stay at 1/4 plus 1/2, so the CDF stays constant.
And at this point, the probability of being less than
or equal to 4, this probability becomes 1.
And so the CDF jumps once more to a value of 1.
Again, at the places where the CDF makes a jump, which one of
the two is the correct value?
The correct value is this one.
And this is because the CDF is defined by using a less than
or equal sign in the probability involved here.
So in the case of discrete random variables, the CDF
takes the form of a staircase function.
It starts at 0.
It ends up at 1.
It has a jump at those points where the PMF assigns a
positive mass.
And the size of the jump is exactly equal to the
corresponding value of the PMF.
Similarly, the size of the PMF here is 1/4, and so the size
of the corresponding jump at the CDF will
also be equal to 1/4.
CDFs have some general properties, and we have seen a
hint of those properties in what we have done so far.
So the CDF is, by definition, the probability of obtaining a
value less than or equal to a certain number little x.
It's the probability of this interval.
If I were to take a larger interval, and go up to some
larger number y, this would be the
probability of a bigger interval.
So that probability would only be bigger.
And this translates into the fact that the CDF is an
non-decreasing function.
If y is larger than or equal to x, as in this picture, then
the value of the CDF evaluated at that point y is going to be
larger than or equal to the CDF evaluated at that
point little x.
Other properties that the CDF has is that as x goes to
infinity, we're talking about the probability essentially of
the entire real line.
And so the CDF will converge to 1.
On the other hand, if x tends to minus infinity, so we're
talking about the probability of an interval to the left of
a point that's all the way out, further and further out.
That probability has to diminish, and eventually
converge to 0.
So in general, CDFs asymptotically start at 0.
They can never go down.
They can only go up.
And in the limit, as x goes to infinity, the CDF has to
approach 1.
Actually in the examples that we saw earlier, it reaches the
value of 1 after a certain finite point.
But in general, for general random variables, it might
only reach the value 1 asymptotically