Welcome everybody, you're watching mr fugu data science. I put together a few common distributions
that i thought were useful with their pdf functions, a brief explanation of each and
a relevant plot to try to understand what's going on. Thanks to the recent subscribers and viewers:
i greatly appreciate it. Let's get started. Here's what you need for the imports if you
want to follow along. Otherwise, go to the link in description and grab the code from
my Github. We're going to start with the binomial distribution. And remember each one of these will
be for the pdf only. So remember a few things; it's a discrete probability function dealing
with independent trials where each trial has an outcome of true or false, success failure, all
right. Here's the mean and the standard deviation. I'm not gonna break this down today, all right
If you want that let me know and i could do separate videos for each one of these functions.
So here's the code for the plotting, i'll let you pause this if you want to follow along with
that or get some ideas of how this is put together. Let me explain something here's:
our binomial distribution with n equals three and a fifty percent for a probability or point,
five all right. Looks pretty typical of a normal distribution. Let's see what happens when we start
varying the number of n, keeping with the 0.5 for the probability. We notice something as
n increases; it looks more like a normal distribution, okay. Which would mean that
our standard deviation and our mean will change. Now the poisson distribution is
interesting. It's a discrete distribution dealing with independent events with some
expected number of events and some time interval lambda, okay. Here's what we're dealing with for
the mean and the standard deviation. Now you could use this in a few applications: such as these
if you want to read that. But, let's get into what the plot looks like to get a general idea
so we can see that if we're dealing with lambda that is actually our mean for 1 000 samples with a
poisson distribution. We will notice that it looks somewhat like a normal distribution and skewed,
okay. You'll notice this skewing. Now what happens if we start playing with this? We'll notice that
there's a trend with our lambda. As you increase your mu, you'll notice that this starts becoming
a normal distribution with a tail. So as we increase our mean starting with 1 and going down
to 10 our sample will end up looking more like a normal distribution. That's the takeaway from this
okay. Wow what is this Weibull distribution that we have here? It's continuous, it's used for time
to failure events: given that you've survived thus far. The first thing that you should notice here
is that at the beginning you should always expect some failures. This stuff is going to seem pretty,
you know like common sense. If you think about it. You're going to have some constant amount
of failures and as you get older in life or more time goes on you have more chance of a failure. So
if you think about this in terms of your life: it's pretty relevant, okay. And you can relate
this to your hazard function. I'm not going into that today. But, you could look at what's going on
with that because it's very relevant in the real world. Here's some of the use cases that we have
for this if you're interested . Now, you have two parameters to think about: your shape which
is your alpha and your scaling which is your beta. And then you just have some time for your quantity
to fail, which is what you're interested in. Now here's three particular scenarios: if you have
alpha equal to one then you have a constant rate of failure. If it's greater than one the rate of
failure is going to increase over time. And then the opposite if it's negative and that would mean
that you have like defective items. Now here's the mean and the standard deviation, if you care.
And you could pause this if you're interested in plotting this out or seeing what was done here.
The first thing varying the scaling while retaining the same constant shape. And you notice
the same thing that we're having in the other ones. You have this lowering, which would look
like the mean is changing and it's becoming more broader. And looking somewhat normal with a skew
okay. The next thing we'll look at is changing the shape. Here's the code for that.
Now when you're changing the shape; you'll notice that as you increase the shape you're going to
also increase the proportion of failures that you have. And what's going on here ? It looks
like it's becoming more normal and this tail that you have on the right where my cursor is decreases
immensely. So it looks like it's becoming somewhat normal. Let's look at some very specific cases.
For instance, if we're looking at the values of the shape between 3 and 4. We'll notice that
it's becoming more like a normal distribution. But, in this case you're going to be more prone
for failures. It's becoming more normal. But you'll notice that it's becoming more prone to
failures at the end of its life. So take that into consideration. The next thing that we're going to
look at is, let's say the shape was two. You see this skewed data going to the right. And you have
to understand that the interpretation is saying that you have a sharp increase in the number
of failures and then over time it decreases, all right. Now, what if this failure is less than one?
Between zero and one. What happens? Well you have this; the smallest number here. Which would be
point one. You have this extremely high drop off, and it looks like you're dealing with an
exponential function. Then you increase this to 0.5 and it's not as steep. But, you still have
this drop off. Then you get to 1 and it's looking like it's you know moderately sloped, alright.
That's something to denote and this would suggest that you have a high rate of failures in the
beginning. Okay, now we're going to deal with the gamma distribution. You have continuous data, you
have an alpha which is your shape, and your beta which is your rate parameter. And let's look at
some of the uses, okay. We're going to deal with this one right here. Since, it's an alternative
and it's more readily used. There's some use cases that you could think of; and here's the mean and
variance for these. Let's get into the graph to figure out what's going on here's. The code for
that if you're curious you notice this, right here. This looks like an exponential function
once again. The takeaway from this: if you're increasing your mu or for us that would be k for
the function above. If you pause the code as it increases you have a flattening out. If in the
case of this exponential, alpha equals beta which k equals mu at one, that's when you end up getting
this situation here. So if both of them equal one, then when you start raising this. let's look at...
Let's make this a little smaller, notice something here's. When they both equal one, two and one,
two and two, three and three and five and five. Okay, let's go back in here real quick and
notice we'll have this really sharp peak. Okay, when this mu is a larger value than our actual
scaling. Then we'll notice that as we increase that value we'll have a lowering or flattening
of this. And then as we increase the scaling: we'll end up having a more broader looking graph.
If i increase the mean value this would start increasing, okay. Now we have the f distribution
which is continuous. There's two types of degrees of freedom that are used in this function and it's
not symmetric about the mean okay. Some common uses would be checking for the difference between
two variances or using it for anova or some type of inference, okay. And you have to define these
as positive values as well. Let's look at some tips real quick. So if you have a large variance,
put it in the numerator you can have a right tail or a skewing to the right. If you're using two
tails then you need to divide by two. The standard deviation will need to be squared to get variance,
and you have to watch out for type two error, okay. So if your degrees of freedom aren't
given then use a critical value from a table, all right. Here's the mean and the variance for this.
Let's scroll down a little bit. Here's the plotting that was done with this
okay. And you'll see this highly skewed to the right. That's fine. Just change some of
the parameters and see what happens with this so we get an idea. As you increase the degrees of
freedom for both: such as this. You see degrees of freedom one, degrees of freedom 2. Soon as
we start increasing both of these, you'll notice that an approximation of the normal distribution
occurs, okay. Let's scroll down a little bit so you can see what this looks like. And here we go.
At various degrees of freedom: as we increase the degrees of freedom you'll notice that it becomes
more like a normal distribution. And this tail, as usual you'll see that it decreases the area under
the curve here. And you still have this long bit. But this is what you need to pay attention to.
And this right here. The next thing that we're interested in is the chi-square distribution. This
is going to be something with skewed to the right, and continuous just like our other skewed data.
Here's a few use cases that you can have where you're looking for either the estimation of your
confidence intervals, checking for independence, finding differences between observed and expected,
or looking for a goodness of fit test. All right, now let's scroll down
and look at this okay. We're increasing the degrees of freedom and you'll see that
the mean decreases. You're becoming more normal and it's also changing your standard deviation,
as well. But, you'll still notice right over here you still have this long right skewed tail,
okay. Also you'll notice something most books will tell you: hey, if you have 30 more degrees of
freedom just consider it as a normal distribution. You'll see this noted in books and people in
in youtube videos. So just be aware that that's just something that's commonplace that people say
okay. The student t-test is next. And this is continuous and used with hypothesis testing to
estimate for example the mean of a distribution with some small sample size, or if you're looking
for confidence intervals. The takeaway: as you increase the size, you get more closer to a
normal distribution. And you have to remember that you have n minus 1 degrees of freedom,
by the way. So scroll down and notice a couple things: first right here. If you're looking to
see if two populations are the same with respect to some variable. That's what's going on. Okay,
as usual you increase the number of degrees of freedom we approach a normal distribution.
Okay fine. You have to consider you're trying to look for data that doesn't have outliers and not
a lot of skewness, okay. And usually you don't know your population . Instead you're doing an
estimation, using your samples. That's something to remember. I know it's elementary but, if you're
new to this these are just some hints and tips that are going to help you get through this, okay.
Scroll down and visually depict what's going on. As we increase the degrees of freedom:
we get to this red one right here. It looks more normal as always pay attention to your
confidence intervals, how those change the standard deviation and how your mean changes.
Logistic regression we're dealing with something that's continuous, great for categorical data,
good for looking at binary classification. For instance let's see what's going on if you're
looking for something that's like: chemical reactions or growth processes. This is also
good but here's what the plot looks like. If you're curious for the code this doesn't look
very useful. It's just showing where confidence intervals are cut off for some something that
looks normal. Let's look at doing a comparison and change up our mean and standard deviations and see
how this changes. So we're going to click on do our link that's based on wikipedia, and let's
scroll out just a little bit. You'll notice that here's our mean, and here's our standard
deviations that are changed as usual. We're going to start with here, this is our furthest left with
the smallest mean the smallest standard deviation and we're starting to increase going this way
okay. So here's where this dark blue is. We have a 5 for the mean but, a 2 for standard deviation.
And then we'll end up having a 6 with a standard deviation of 2. And then we have a mean of 9 but,
we have a standard deviation of 3. The same mean but, a larger standard deviation and
that's where you see this flattening right here, okay. That's a visual depiction of
getting the pdf for the logistic distribution. Let's go to our last one which is the pareto,
pareto. However it's pronounced. And it's a power probability distribution, that is continuous
and you have your alpha which is your shape and your beta which is scaling. same as usual.
Here's some common use cases in the real world, okay. This is really good if you're looking at
like stock prices dropping, or anything like natural phenomenon. Things like this, you know.
So there's a lot of code for this one; not sure if you want to pause. But feel free. Let's just get
an idea as you increase your lambda and your k. In this instance you can say that your alpha which is
going to be your slope, and this would be the k for this process gets a steep drop off as you
increase it okay. So this is increasing, and it gets a steep drop off. And your scaling is just
moving up to the right. And we can think of this as most people have no money and very few people
have a lot of money: like wealth distribution. You could think of that in this case for instance:
people who are making you know to say about right here a hundred thousand dollars or more
the scaling would be different for your your x and your y. But it would still have this kind of drop
off and that's basically the the gist of what's going on there. Well it looks like
it's a variation of the exponential function, okay. So as always i'd like to say please like,
share and subscribe. And if you subscribe turn on that notification bell. Don't forget to hit
me up on instagram or twitter and i established a, buy me a coffee. Because, it takes a long time to
make these videos. So please help me support this channel and thank you for watching. Bye