Welcome everybody, you're watching mr fugu data science. Today, we're going to focus specifically
on the binomial distribution. As always i'd like to say thank you to the recent subscribers and
viewers, i greatly appreciate it. Get a hold of me on instagram and twitter. Also, I set up buy
me a coffee: because, it takes a long time to make these videos. So please help support this channel,
i'd appreciate it. Thank you. Now, you only have a couple imports today. You can follow
along with me and pause the video or you can go on my Github which will be in the link in
description to download today's code. The binomial distribution can be thought of as the sum of the
Bernoulli trials, which end up giving you the binomial distribution, okay. So if you're doing
like a heads and tails, success fail situation and you have a large number of trials that's where you
end up with the this distribution. The pdf can be explained by this formula, right here. We'll
simplify explaining this below. You have your mean and your standard deviation here. If you look at
the cursor the simplified version of thinking about this can stem from four things; each of
your outcomes is independent for each trial. You have a fixed number of trials and the probability
of success for each trial is the same. And the only two outcomes you have is true false success
fail, okay. The things you should know from this the one minus x deals with your fails,
the p of x is your probability of the number of successes for each trial, and your x is
your success. Your x is the number of successes in each trial and p is your probability of success.
And n is just the number of trials. Now let's see what happens when we vary the number of trials,
okay. Keeping everything else constant that's what ceteris paribus means. Now we're going to have the
probability of 0.5 and we're going to vary the number of samples we're going to have for each
trial and let's see what happens. If you pay attention: as we increase our samples from 5
to 100. You notice that you approach a normal distribution. Your mean which is your peak for
each one of these shifts because, now you have more samples to basically do a comparison against
for your given probability. So you have more of a chance of landing on something
over here. If you have more times to achieve it basically. Then we have our cdf and this is the
function for that. And if you notice right here, this is the floor function for k inside of here.
Okay so let's understand what's going on. You're basically finding the probability of obtaining
the probability you observe less than or equal to some x number of successes and n trials
with some given probability. So just remember it's less than or equal to the number of successes
that's all you have to get from the takeaway of that. We'll see what this looks like in a
plot soon. So you get more of a definitive idea. Now some tips that you can look at when you're
evaluating this plot. You're going to notice that it has an s curve. The steeper it is, the
smaller the width of the curve for instance. If it's almost a vertical curve, you'll have a high
amount of kurtosis. Which means that if you are plotting this and looking at it as a density plot:
you're going to have some really fat tails, okay. You also get something that looks more normal, as
that curve starts approaching 45 degrees okay. And if you were plotting two curves and they looked
identical but they were shifted it would just be in a multiplicative effect. You just have some
variable shifting it. Like if you had two parallel lines that were the same and you just had some
variable that was a multiplier. Okay the other thing that i need to mention is that if you have
a lot of trials and it's fairly symmetric for your success failure rate; then you can approximate
the distribution itself. But, you need a lot of trials for that. We're gonna look briefly at two
cdf plots real quick and you're gonna see what's going on. I decided to first start with looking at
a probability of 15 followed by a probability of 40 given that we have the same amount of trials.
Let's scroll down and we'll notice that the first one with probability of 15 approaches
one fairly quickly at about nine you're already at 100. So you would interpret this like this,
you would say that for any value of k for our success would be summed up as a cumulative sum
of anything that happened prior to it. For instance, if this was; let's say that this
was ninety percent. We would say look, we have a 95 percent probability of success
given any value is at least six. Because it's summing everything that's previous. Then we'll
look at the same thing. But, with a probability of 40 and you notice that it's going up to about
17 would encompass 100 of our data. And then everything else is just really irrelevant.
Now let's notice the relationship between the pdf and the cdf. If we were looking at exactly eight
so we would see that the pdf would need to sum everything until we reached eight. In order to get
this one particular line for the cdf, representing the same thing. So roughly it's like 70, something
right. If you summed all of these together on this pdf you get this one line right here which is this
representation. That's the only distinction if you're trying to use this for interpreting. These
are the available functions that you have with in r. And the one thing that i want to mention that's
of great importance here: is this lower tail. If it equals true, you're going to the left. If it's
false you're going to the right, okay. Now the d for the density is returning your pdf. The p
is returning basically your cdf that we just looked at above and you have to notice that you're
interested less than or equal to or greater than or equal to for your trials of q, alright. That's
something you need to pay attention to. That's what that should be changed to actually. You have
your q, which is dealing with your quantiles and that's letting you to predict a number of
successes. So you throw in some quantile like the lower ten percent. You do point ten, where
you say like the upper sixty percent point six something like this right or the upper ten percent
point nine . You chuck all that in with your probability of success and whatever your trials
are and it spits out a number which is a predicted number of how many successes you'll have with that
data. It's the opposite of using the pbinom okay. And instead you're getting a less than or equal to
the size that you're chucking in. That's something you need to pay attention to okay. I didn't put
anything for this because you're just creating a random distribution and we're going to see this a
lot when we're plotting this out. A few things you should pay attention to, and i'm going to mention
this specifically in the plot. You could pause this and read it but, i'm going to mention it when
we look below. So let's very some probabilities and see what happens with these data.
We're going to look at a probability of twenty percent, fifty percent,
seventy percent given that we're going to have 40 trials. And we're just going to use this random
one and do it a thousand times. Let's see what happens. So i used a random seed and this is what
we got with this specific random seed. When you just keep running this, these are going to change
because you're using r binom. So just be aware of that. Yours is going to look different than mine.
The first thing that i want to mention is this: as a probability goes from left to right from 0.05
the curve will skew to one side or the other for the p. Okay, the next thing i want to suggest is
that for the same number of successes but altering the probability you'll notice that you have this
shifting okay. So we go from lower to higher probabilities and you'll see that the mean
for each of these causes it to shift because, basically as you have more opportunities to
achieve a higher probability. For instance, if i had more successes available to me and i knew
that for each success i had a high probability that means that this probability is going to now
increase. You understand what i'm saying. That's the take away from that. And let's figure out
what some of these numbers are doing. So these are the same things that we did for this plot
above so you can get a numerical value so we can kind of go up and compare this, okay. So you can
see that we have about 8, 20 and 27 for the mean for each of these probabilities for our simulated
version with the random binomial. So you got about eight got like 19, 20 and then about i don't know
28 or so. It looks like it's lining up somewhat and this is gonna this is going to lop over to one
side or the other just depending on how you're running this without a seed. So i printed out
here now notice that these are the simulated values. Because, these were relevant to the
randomness that we created above. But we can also get the exact values from these as well.
Here's the actual values that we're going to look at real quick. But let's notice something.
What is really going on here? For instance, if we had 40 items that had a percent 20 chance of
occurring. What was the probability of exactly getting eight? We ended up with 15.6 percent.
So i ran this for each one of the exact values so we can get a comparison and see what is actually
going on. You are going to have some discrepancy between the simulated and the actual values:
take note of that, all right. Chance of finding exactly eight assuming that we're going to have
a probability of 20 of our 40 people sampled for each trial who really like coffee and let's see
what the probability of success would be if we had exactly eight people out of 40 who liked coffee.
Feel free to pause this if you want to write this down or figure out how to decipher this.
But here we go. Out of 40 people we wanted exactly eight people exactly eight people.
We have a 16 percent chance that they like coffee given that the trial had 40 people.
And we knew that there was a 20 chance that they would like coffee. Now let's look at less than or
equal to eight people. Or in this case we're looking at the left tail and we have to say
lower tail equals true here, okay. That's what's going on with this. And if you notice here we're
using the pmf and the cdf so we're able to plot this out okay. So just pay attention to that
because, we're looking at the less than or equal to. So remember cdf okay. We have our less than
or equal to, and it turns out at eight we have 59 okay fine. Which means that we'll have a 59.3
probability of obtaining less than or equal to eight given for each trial we have 40 whatever
it is with a 20 chance of whatever it is occurring okay. Here's the simulated value versus the exact
value. If you keep running this value you're going to get different values each time you run it,
just be aware of that. Unless you run a seed. Then let's do an upper tail which is your right side,
meaning that we need to now do lower tail equals false and let's use this problem. It's just a joke
of course. Assume we have a group of chickens we want to catch. But they're really fast and
say we have 40 chickens and we have a 20 chance of catching them with a whistle that attracts
them. What's the probability of catching at least eight chickens? Meaning we want eight or more,
okay fine. We gotta remember something that's really important here, okay. You're gonna need
to do one minus your q value okay. If you're doing greater than otherwise if you're doing
strictly greater than you could just use q. Now let's get into this. When you're doing the plot,
you're gonna have your q equal to negative one and then one minus whatever the max value is so you
can establish this number. Because, if you leave it at 0 to 40; what you're really going to get is
a strictly greater than value. Which is not what you want. What you can do is keep this but, lower
this value to 7 instead. That's another option you can do so you'll notice that if we're looking at
this right tail. And we want eight greater than or equal to finding our little chickens. We got a
50 percent chance of obtaining this, given that we had a 20 chance of catching a chicken with a trial
of 40. And here's the simulated values. And here's another way that you can do this same thing:
to do like a self-check. That's what i had to do because, i was like what the hell is going
on here when i first did this. Because, it was kind of odd. I said if i'm taking the left tail
and i'm doing one minus that i'm going to obtain everything else, okay. And so i had to do by seven
and not eight for each of these to line up. And then this is still the same. Now we can get into
our strictly greater than. And i did this where i could have instead kept this formatting of 0
to 40. But, i could have also just changed this value instead of 8 i could have increased it.
And that would have worked as well. So that's just a side note. Let's assume that we have 20 percent
of adults that are allergic to bathing due to allergic reactions. Assuming they are given magic
soap, there are 40 people given the medication. What is the probability that the medication
is effective in more than eight people we study. Because, you know sometimes people just don't like
showering and they need magic. So of course i'm joking, or am i? Now, i did the same goofy thing:
just pay attention to that. And let's look at this. Let's scroll in here and we'll notice that
right here: you have to pay attention to this. Because, if you're doing your plotting wrong
it'll be over here on your eight. So just pay attention because you want greater than eight,
okay. We got a 41 percent chance given that we provide some magic soap to these 40 people;
that nine or greater of these people will respond favorably to using it. Considering
there is a 20 percent chance that they want to use the soap in the first place, okay. Then
we got our same little digits here. But let's get into some considerations. The pdf is great if you
have tag words: such as exact, sum, some specific number. And then cdf has tag words : like,
no more than, at most, fewer than less than, etc. We have a quantile that we can use and i'm not
going to give an expression here because, you're returning an actual number. A predicted number. So
let's assume: i have 40 people who fall in the bottom 10 percent of decorum with a 75
chance of being weird and each time we test 40 people. That means that we have zero to 26 people
of 40 who fit this category. And if you're not sure what this is:
that's me just joking around again. Now the decile is something that's interesting. You're
evaluating: do the sub-sample variations change over different sub-samples? What the hell does
that mean? Let's take the thing from above. Let's assume that we were looking at those 40 people
with a 75 percent chance of not having decorum. And we're going to throw them in the lower 10,
lower 10th quantile or you could you could say first decile i guess is how that would
work out . And what's happening is, it's splitting this into the 26 to 40 that we had.
But it's not splitting them based on like 26 through like 40. You know you see this
range it seems kind of odd here right. But, just look at this you're basically deciding:
hey this is the numbers from 26 to 40 on the y-axis and on the x-axis you have your your
proportion. And you're evaluating with this s-looking thing. Hey how much is this changing
relative to the other points for this trend. I'm not really familiar with this type of analysis;
i stumbled on it. It's really good for academic stuff, investing economics and stuff like that.
And i left a page that you can go to and look at this to evaluate. But that's just
not something i'm familiar with. So i'm not going to do some interpretations on this. But,
as always please like, share and subscribe. And if you subscribe turn on that notification
bell . Don't forget to hit me up on those socials. And if you can and you like the material hit me
up and buy me a coffee. I greatly appreciate it. Have a good one see in the next video.
Bye