Hey guys, in this video I'm going to go over a statistical technique called the
inverse transform method. And this is basically a method of generating
non-uniform random variables with a desired distribution F and it's a pretty
simple technique that I'm going to go over in a minute
but basically what it lets you do is it lets you create random variables that
are not uniformly distributed and that have some desired distribution that you
want. And from there, you can use this technique to randomly sample from that
random variable that you generated. I'm going to do an example in this video
of generating a random variable that's exponentially distributed with this PDF
and I'm going to go over how you do that using the inverse transform method and
then I'm going to go to R and do a quick demo of how you can create random
samples with this desired distribution, in this case again using the exponential
distribution as an example. So let's go ahead and get started. I have the lemma
for how the inverse transform method works written out here and basically
what it does is it takes U, a random variable that is uniformly distributed,
and it takes a CDF of a distribution F that you desire your random variable to have
and using the inverse of the F distribution that you want, you just plug
in U to the inverse CDF and then if you sample from this distribution, it will
have the desired distribution that you want. So if X is a new random variable
that you generate and it has the inverse CDF with U plugged in to it, that will
generate a random variable X with a new distribution that you desired which is F.
Because you have to get the inverse CDF as part of this process, sometimes when
you take the inverse of a CDF there's not one unique inverse, so sometimes you
also have to have this additional definition which is that F inverse is
equal to the infimum x such that F of x is greater than or equal to u. So if you
have more than one possible solution for the inverse this can help you get a
unique solution and still calculate the CDF or the inverse CDF
using this technique.For this particular problem, I don't have this problem so I
don't have to use this additional technique for the example here. So I'm
going to go ahead and get started with this example. For this example, I
basically want to use the inverse transform method to create a random
variable X that has this PDF distribution and I'm telling you if you
can't recognize this that this is going to be the exponential distribution, so I
want to be able to generate a random variable X that's distributed
exponential with the parameter lambda. This is the PDF for this distribution.
Let's go ahead and go over how we do that using the inverse transform method.
First, this is the PDF of the exponential distribution. If you're given the PDF,
the first thing you have to do is find the CDF or the cumulative distribution
function. That's just going to be the integral from zero, which is the
beginning existence of x here, zero to x. And I'm just going to plug in the PDF,
which is lambda e to the negative lambda x dx and solve for this integral. So I'm
going to use u substitution and say u is negative lambda x then du is equal
to negative lambda. I have lambda here already, so I just have to plug in
negative one. So I have negative one times lambda e to the u and then this is
basically du. So the integral of that is going to be negative e to the negative
lambda x from zero to x. When I plug in x and zero, that's going to give me
negative times e to the negative lambda x minus e to the 0 e to the 0 is 1 so
that's going to be 1 minus e to the negative lambda x. The CDF for this
PDF is going to be capital F of x is equal to 1 minus e to the negative
lambda x for x is greater than or equal to 0 and it's going to be 0 for x is
less than 0. And this is the CDF of the exponential
distribution. Once we have the CDF that we want that's going to be our F and
then the next thing have to do is find the inverse CDF, so F
inverse is what we're going to try to find next. So for step two, I'm just going
to say that F of x is equal to y and y is equal to one minus e to the negative
lambda x and then to solve for the inverse, this is just like an algebra
problem. I'm basically going to solve for x. To do that I'm going to subtract
one from both sides then multiply by negative 1, so that's 1 minus y is equal
to e to the negative lambda x. I'm going to take the natural log of both sides.
That gives me negative lambda x on the right hand side and then I'm going to do--
multiply by negative 1 on both sides and divide by lambda on both sides.
That gives me that x is equal to negative log 1 minus y divided by lambda.
This would be F inverse y is equal to negative Ln 1 minus y over lambda.
The third and final step, which is very straightforward, is we just have to plug
in the random variable U into that function. So the third and final step is
that X is going to be equal to F inverse U so this would be negative Ln 1 minus U
over lambda where U is a uniformly distributed random variable from [0, 1]
inclusive. This is basically our final solution and if we sample from a random
variable with this distribution, we will get X that's exponentially distributed
with parameter lambda. So I'm going to now switch over to the R demo and I'm
going to generate 1000 random samples or really pseudo random samples that are
exponential lambda and for this demo I'm going to say lambda is equal to 3.
I switched over to R and I'm just going to run this short little demo to show
how you can randomly sample from a non-uniform distribution using the
inverse transform method and it's a very short line of code. If anyone wants to
replicate this, I'm just going to set the seed to 1337 just to make sure I get
the same numbers over and over again and I'm going to
you try to randomly sample 1000 samples. I'm going to try to generate 1000
samples in this demo. The first thing is I'm going to randomly sample from U,
the uniformly distributed random variable. I'm going to get 1000 random
samples from the uniform [0,1] distribution and I'm setting that to a
vector variable a. So now if I choose to view a or print a, it's going to give me
1000 samples that are between the values 0 and 1--approximately uniformly
distributed because it's randomly sampled. In the next line of code, I'm
going to plot a histogram of the vector a and that should give me roughly a
uniform distribution. So you can see all the different values that a is taking on
in the vector and these are roughly uniformly distributed across all the
possible values 0 and 1. We do have a uniform random variable in a. Then in the
next part, b is going to be the new F inverse distribution that we created.
We said that that was negative the natural log 1 minus u in which case u is
a here divided by 3 that's the parameter lambda that I'm setting equal to 3.
I'm going to run that. That will give me 1000 random samples from the new
distribution and we're saying that using this inverse transform method, that new
distribution is exponential with lambda is equal to 3. If I print B this gives me
all the new values of the random sample. Now we have 1000 new random samples
that are not uniformly distributed. And again to prove that it kind of looks
roughly exponentially distributed, I'm going to plot the histogram of all the
values b. And now when I plot this, we see that we went from a roughly uniformly
distributed sample to a roughly exponentially distributed random
variable b. So it looks like the inverse transform method works, at least
empirically it seems to work, and we can say that we have 1000 samples that are
distributed exponential with lambda equal to 3. And that's it for this video
and thank you guys for watching!
*chiptune music*