In this segment, we start a discussion of multiple
continuous random variables.
Here are some objects that we're already familiar with.
But exactly as in the discrete case, if we are dealing with
two random variables, it is not enough to know their
individual PDFs.
We also need to model the relation between the two
random variables, and this is done through a joint PDF,
which is the continuous analog of the joint PMF.
We will use this notation to indicate joint PDFs where we
use f to indicate that we're dealing with a density.
So what remains to be done is to actually define this object
and see how we use it.
Let us start by recalling that joint PMFs were defined in
terms of the probability that the pair of random variables X
and Y take certain specific values little x and little y.
Regarding joint PDFs, we start by saying that it has to be
non-negative.
However, a more precise interpretation in terms of
probabilities has to wait a little bit.
Joint PDFs will be used to calculate probabilities.
And this will be done in analogy with
the discrete setting.
In the discrete setting, the probability that the pair of
random variables falls inside a certain set is just the sum
of the probabilities of all of the possible pairs inside that
particular set.
For the continuous case, we introduce
an analogous formula.
We use the joint density instead of the joint PMF.
And instead of having a summation, we now integrate.
As in the discrete setting, we have one total unit of
probability.
The joint PDF tells us how this unit of probability is
spread over the entire continuous
two-dimensional plane.
And we use it, we use the joint PDF, to calculate the
probability of a certain set by finding the volume under
the joint PDF that lies on top of that set.
This is what this integral really represents.
We integrate over a particular two-dimensional set, and we
take this value that we integrate.
And we can think of this as the height of an object that's
sitting on top of that set.
Now, this relation here, this calculation of probabilities,
is not something that we are supposed to prove.
This is, rather, the definition of
what a joint PDF does.
A legitimate joint PDF is any function of two variables,
which is non-negative and which integrates to 1.
And we will say that two random variables are jointly
continuous if there is a legitimate joint PDF that can
be used to calculate the associated probabilities
through this particular formula.
So we have really an indirect definition.
Instead of defining the joint PDF as a probability, we
actually define it indirectly by saying what it does, how it
will be used to calculate probabilities.
A picture will be helpful here.
Here's a plot of a possible joint PDF.
These are the x and y-axes.
And the function being plotted is the joint PDF of these two
random variables.
This joint PDF is higher at some places and lower at
others, indicating that certain regions of the x,y
plane are more likely than others.
The joint PDF determines the probability of a set B by
integrating over that set B. Let's say it's this set.
Integrating the PDF over that set.
Pictorially, what this means is that we look at the volume
that sits on top of that set, but below the PDF, below the
joint PDF, and so we obtain some three-dimensional object
of this kind.
And this integral corresponds to actually finding this
volume here, the volume that sits on top of the set B but
which is below the joint PDF.
Let us now develop some additional understanding of
joint PDFs.
As we just discussed, for any given set B, we can integrate
the joint PDF over that set.
And this will give us the probability of
that particular set.
Of particular interest is the case where we're dealing with
a set which is a rectangle, in which case the situation is a
little simpler.
So suppose that we have a rectangle where the
x-coordinate ranges from A to B and the y-coordinate ranges
from some C to some D. Then, the double integral over this
particular rectangle can be written in a form where we
first integrate with respect to one of the variables that
ranges from A to B. And then, we integrate over all possible
values of y as they range from C to D.
Of particular interest is the special case where we're
dealing with a small rectangle such as this one.
A rectangle with sizes equal to some delta where delta is a
small number.
In that case, the double integral, which is the volume
on top of that rectangle, is simpler to evaluate.
It is equal to the value of the function that we're
integrating at some point in the rectangle --- let's take
that corner ---
times the area of that little rectangle, which is equal to
delta square.
So we have an interpretation of the joint PDF in terms of
probabilities of small rectangles.
Joint PDFs are not probabilities.
But rather, they are probability densities.
They tell us the probability per unit area.
And one more important comment.
For the case of a single continuous random variable, we
know that any single point has 0 probability.
This is again, true for the case of two jointly continuous
random variables.
But more is true.
If you take a set B that has 0 area.
For example, a certain curve.
Suppose that this curve is the entire set B. Then, the volume
under the joint PDF that's sitting on top of that curve
is going to be equal to 0.
So 0 area sets have 0 probability.
And this is one of the characteristic features of
jointly continuous random variables.
Now, let's think of a particular situation.
Suppose that X is a continuous random variable, and let Y be
another random variable, which is identically equal to X.
Since X is a continuous random variable, Y is also a
continuous random variable.
However, in this situation, we are certain that the outcome
of the experiment is going to fall on the line
where x equals y.
All the probability lies on top of a line, and
a line has 0 area.
So we have positive probability on the set of 0
area, which contradicts what we discussed before.
Well, this simply means that X and Y are not jointly
continuous.
Each one of them is continuous, but together
they're not jointly continuous.
Essentially, joint continuity is something more than
requiring each random variable to be continuous by itself.
For joint continuity, we want the probability to be really
spread over two dimensions.
Probability is not allowed to be concentrated on a
one-dimensional set.
On the other hand, in this example, the probability is
concentrated on a one-dimensional set.
And we do not have joint continuity.