Hey guys, in this video I'm going to do this problem.
We're given variables that come from a Poisson distribution
and we're asked to use the Rao-
Blackwell theorem to estimate the probability that the Xs are equal to zero, which is equal to e to the negative lambda.
Let's go ahead and get started. The Rao-Blackwell theorem has the following formula. You can write it as
delta star T of X is equal to the expected value of
delta of X
given T of X is equal to T.
Basically the idea with Rao-Blackwell is you're going to take an estimate of whatever you're asked to find--
so we're asked to estimate e to the negative lambda--
we're going to plug in some
estimator, known as the original estimator, and a lot of the times--well sometimes you're given
what estimator to plug in and it might be just a really bad one. And
through Rao-Blackwellization you're going to find a better estimator.
Why is it a better estimator? And also you're going to be conditioning on a
sufficient statistic. T of X equal to T is some sufficient statistic. The Rao-Blackwell estimator is a better estimator
because of its two properties that it has.
The first property that it has is that it's considered a minimum variance estimator. The variance of delta star
T of X is always going to be less than or equal to the variance of
delta of X. Whatever estimator you start with, the one that you end up with is always going to have at least
that small of a variance as you started with or many of the time
it's gonna be less variance. And another property is about the bias. The expected value for
this, whether or not delta star T of X is biased, depends on the original estimator.
If you plug in for your original estimator an unbiased
estimator for whatever you're looking for, then the Rao-Blackwell estimator will also be unbiased.
Similarly, if you plug in a biased estimator for your original estimator, the Rao-Blackwell
estimator will be biased. If you plug in biased, you get out biased and if you plug in unbiased, you get out unbiased estimators.
Usually you'll learn about Rao-Blackwell first and then later your you'll learn about uniformly minimum variance unbiased
estimator and that's a class of estimators or sometimes
it's just called minimum variance unbiased estimator. And so one way of finding the minimum--
and these are good estimators obviously--one way of finding this category of
estimators is by plugging in through the Rao-Blackwell theorem an unbiased estimator.
And so when you plug in an unbiased estimator, you'll always get as the output the minimum variance unbiased estimator.
That was a little background. Now
let's go ahead and do this particular problem.
You just have to plug in two things to solve it. The first thing I'm going to do is get my original
estimator.
For this one--
for these types of problems, a lot of times you can plug in what's called the empirical CDF. That will look something like this, which
is just an average of all the values--in this case
this is a discrete distribution--so the average of all the values of an indicator function.
That kind of models
empirically a CDF. In this case,
I'm just going to do the first value and say that my estimator is that if X1 is equal to 0,
that's an estimator of P(X = 0). It's not necessarily a good estimator
but it doesn't have to be because we're going to put it through the Rao-Blackwell formula to make it better anyway.
And then the other thing that you need is this sufficient
statistic. To get this, you could--if you don't know it already when you're starting out a problem--
you can do maximum likelihood for example. If you do something like this,
you'll end up getting that it's the sum of Xi
or you can just know that already. In my case,
I already just knew that.
Now we just have to plug in and kind of simplify to get the estimator that we want.
I'm going to go ahead and plug in our
delta of X and T of X into the formula. That's going to give us E of X1 is equal to 0
given that the sum of Xi is equal to T.
This is equal to the probability that X of 1 is equal to 0
given that the sum of Xi is equal to T.
This is just a conditional probability now.
We can
use the formula for conditional
distributions or conditional probability. That is equal to the probability of X1 is equal to 0 and
the sum of Xi
is equal to T over the marginal, which is this part, which is the probability of the sum of Xi
is equal to T given independence.
Sorry this should be i is equal to 2 to N.
If we know that X1 is equal to 0, that means that the sum from i is
equal to 2 to N, so the rest of the Xis
has to be--the sum of that--has to be equal to T because we know that the sum of Xi
has to be equal to T.
If the first value is 0, that means the rest of the value is minus the first value has to be equal to T.
We have
independence in the numerator.
With independence, the joint is equal to the product of the marginals. Again we can simplify that to
P of X1 is equal to 0 times the probability that the sum of--the sum from i
equal 2 to N of Xi is equal to T
over the probability that the sum of Xi is equal to T
and the denominator is for all Xis not just from 2 to N. The Poisson has a
property where if you have Xis
that are distributed Poisson lambda and Ys that are distributed Poisson lambda,
then the joint is going to be equal to Poisson 2 lambda. Basically when you have multiple
Poissons, you just add them as your parameter. With this property we can simplify
Again, and be--well, we can basically plug in now. What the
distribution or what the estimator will look like. Given this information
for the first probability, we can plug in now that this is just equal to e to the negative lambda. For this,
using this property we know that this---this will be N minus 1 values.
We're going to plug in N minus 1 lambda as the distribution
parameter. That's going to give us a negative N minus 1 lambda times N
minus 1 lambda to the T over N minus--Sorry over T
factorial. That's the numerator and then the denominator is going to be N values.
We're gonna have--that's distributed N lambda. The denominator is going to be e to the N
lambda, N lambda
to the T over T factorial.
Now we just need to simplify. The T factorials will cancel out. We can cancel out the lambdas here. If we distribute this,
this top part is going to be equal to e to the negative lambda. These
cancel out and we're left with N minus 1 over N
to the T. T is equal to the sufficient statistic. It's the sum of Xi is equal to T.
We can plug in for our Rao-Blackwell estimator as our final answer N minus 1 over N to the
sum of Xi
and this is our final answer. At first, it might look a little bit weird
but if we simplify this, this is actually 1 minus 1 over N to the sum of Xi
and we can further simplify that as 1 minus 1 over N times N--
to the N times
the sample mean. And this limit converges to e to the negative
sample mean. Since we were asked to find an estimator for e to the negative lambda,
it makes sense that we have this as our final answer because this--the limit converges to e to the negative
sample mean.
And so we're done with this problem.
*chiptune music*