We will now go through an example that brings together
all of the concepts that we have introduced.
We have a stick of length l.
And we break that stick at some random location, which
corresponds to a random variable, X.
And we assume that this random variable is uniform over the
length of the stick.
So its PDF has this particular shape.
And for the PDF to integrate to 1, the height of this PDF
must be equal to 1 over l.
Then we take the piece of the stick that we are left with,
which has length X, and we break it at a random location,
which we call Y. And we assume that this location Y is
uniformly distributed over the length of the stick that we
were left with.
What does this assumption mean?
It means that if the first break was at some particular
value, x, then the random variable Y has a conditional
distribution, which is uniform over the interval from 0 to x.
So the conditional PDF is uniform.
A conditional PDF, like any other PDF, must
integrate to 1.
So the height of this conditional PDF is
equal to 1 over x.
Are X and Y independent?
No.
One way to see it is that if you change little x, the
conditional PDF of Y would have been something different.
Whereas if we have independence, all the
conditional PDFs have to be the same when you change the
value of little x.
Another way to see it is that if I tell you that x is 0.5,
this gives you lots of information about Y. It tells
you that Y has to be less than or equal to 0.5.
So the value of the random variable X gives you plenty of
information about the other random variable.
And so we do not have independence.
Notice that in this example, instead of starting with a
full description of the random variables in terms of a joint
PDF, we use a marginal PDF and then a conditional PDF to
construct our model.
Of course, with these two pieces of information, we can
reconstruct the joint PDF using the multiplication rule.
The marginal is 1 over l.
The conditional is 1 over x.
So the joint is equal to 1 over lx.
But for which values of x and y is this the correct
expression?
It is correct only for those values that are possible.
So 0 has to be less than y, less than x, less than l.
This is the range of values that are possible in this
particular experiment.
And we can visualize those values.
They are those that correspond to this shaded triangle here.
x and y are less than or equal to l.
And y has to be less than or equal to x.
If you try to visualize the joint PDF, notice that since
it only depends on x not on y, if you fix a value of x and
you look at the slice of the joint PDF, the value of the
joint PDF is going to be a constant on that slice.
On this slice, it's going to be another constant, actually
a bigger one.
On that slice, an even bigger constant.
And actually, this constant is bigger and bigger and goes to
infinity as we approach 0.
Of course, the fact that the slice is constant is just a
reflection of the fact that the conditional PDF is
constant over the range of values that the random
variable can take.
Let us now continue with some calculations.
Let us find the marginal PDF of Y. How do we do it?
Since we have in our hands the joint PDF, we can find the
marginal by integrating the joint.
And in our case, the joint is equal to 1 over lx.
And we integrate over all x's.
Now, what is the range of the integration?
If we fix a certain value of y, the joint PDF is actually 0
in this region and in that region.
So we should only integrate over x's that correspond to
this interval.
What is that interval?
It's the interval that ends at l.
And because this is a line of slope 1, this value
here is also y.
So we integrate over an interval where x
ranges from y to l.
In fact, this is just the range of x's that are possible
for a given value of y.
x must always be larger than or equal to y.
Now, the integral of 1 over x is a logarithm.
And using this fact, we can evaluate this integral.
And it's 1 over l times the logarithm of l over y.
For what y's is this a correct expression?
Well, it makes sense only for those y's that are possible in
this experiment.
And that's the range from 0 to l.
When y is equal to l, we have the logarithm of 1, which is
equal to 0.
So the value of the PDF is 0 here.
As y decreases, this ratio
increases and goes to infinity.
So the log of that also blows up to infinity.
And we get a shape of this form, where the function that
we're dealing with goes to infinity as we approach 0.
Is this a problem having a PDF that blows up to infinity?
Not really.
As long as the area under this PDF is equal to 1, it's still
a legitimate PDF.
And blowing up to infinity is not an issue.
Let us now calculate the expected value of Y. One way
of doing this is by using the definition of the expectation.
It's the integral of y times the density of y, which is 1
over l times the log of l over y.
And the range of integration has to be those values for
which we have a non-zero density.
So we integrate from 0 to l, which are the possible values
of the random variable Y. This is an integral
that's pretty messy.
One can actually integrate it using integration by parts.
But the calculation is a bit tedious.
So let us look for an alternative
and more clever approach.
The idea is to divide and conquer.
We're going to use the total expectation theorem, where
we're going to condition on X. The total expectation theorem
tells us that the expected value of Y is the integral
over all possible values of the random variable X, which
is from 0 to l.
The density of X, which is 1 over l, times the conditional
expectation of Y given that X is equal to some little x.
And we integrate over all x's.
Why is this simpler?
When we condition on X taking a specific value, Y has a
uniform distribution between 0 and x.
And therefore, this conditional expectation is the
expectation of a uniform, which is 1/2 the
range of that uniform.
So we obtain the integral from 0 to l.
1 over l times x over 2, dx.
And finally, that's an integral that we
can evaluate easily.
Or we can think even in a simpler way.
This expression here is the density of x.
This is x itself.
So the integral of this times x gives us the expected value
of X. And there's only a factor of 1/2
that's left out there.
So we obtain 1/2 the expected value of X. But now, X itself
is uniform on an interval that has length l.
And therefore, the expected value of x is l over 2.
And so we get the final answer, which is 1/2 times l
over 2, which is l over four.
This answer makes intuitive sense.
If we break a stick once, the expected value or what we're
left with is half of what we started with.
But if we break it once more, then we expect it on the
average to be cut by a factor again of 1/2.
And so we expect to be left with a stick that has length
1/4 of what we started with.
So this example is a particularly nice one, because
we used all of the concepts that we have introduced--
marginal PDFs, joint PDFs, conditional PDFs, and the
relations between them, as well as expectations,
calculations of expectations, and conditional expectations,
as well as the total probability theorem.