We’re going to talk about a few particular kinds of discrete random variables that describe
situations that have a probability distributions that follow a functional form. The first of
these distributions is the binominal distribution.
The binomial distribution describes situations where we have a series of n Bernoulli trials
A Bernoulli trial is an experiment that has only one of two possible outcomes (hence the
binomial distribution). Examples of a Bernoulli trial could be a coin flip (whose outcome
is either a head or a tail), a game (whose outcome is win or lose), or maybe an exam
(whose outcome is pass or fail). In general, we say the outcome of the Bernoulli trial
is either success or failure. And a binomially distributed random variable provides the probability
of some number of successes in those n trials. Note here that “success” just means “the
outcome that we’re interested in finding the probability of” it may not really mean
a success as a positive thing. For example, if we wanting to find the probability that
we fail two out of the next five exams, “success” in the binomial distribution language would
be “failed exam” not what we would typically define as a success in everyday use.
There are four properties of a binomial experiment, in addition to the typical properties of a
probability: The first is that we are observing n identical
Bernoulli trials. The second is that each trial has only two possible outcomes (naturally,
by definition of a Bernoulli trial). Third, the probability of the two outcomes remain
constant: the probability of a success is p, therefore the probability of a failure
(or the complement of a success) is 1-p. Some textbooks use q as the notation for 1-p, but
I prefer to be explicit. Finally, the trials are independent. That is, observing a success
or several successes doesn’t impact the probability of witnessing another success
or a failure in an individual trial, or vice versa.
Let’s motivate the binomial probability distribution function with the quality control
operations at Duff Brewery. Say a bottle of Duff beer is determined to be either defective
or not defective. Anything out of the ordinary with a bottle, if the label is a little off,
if there is a scratch on the bottle, if the fill of the bottle is a little low, then we
call the bottle defective. Otherwise, it’s not defective. You might imagine that making
a quality control assessment about a bottle of Duff beer involves a Bernoulli trial: there
are two possible outcomes, either defective or not defective, and the bottle can fall
into only one of those categories. Say we take a random sample of five bottles of beer,
and we repeat our Bernoulli trial five times. Random variable X is defined as the number
of defective bottles found in the sample of five. Let’s say the historical probability
of any particular bottle being defective is 0.02. That is, 2% of bottles on average are
defective. We ask a question: what’s the probability that we find two defective bottles
in those five? That is, what’s the probability that X is 2? Let’s enumerate a few possible
ways to get two defective beer bottles out of five.
It could be that the first two of the five are defective. Individually, the probability
that we get two defective bottles then three non-defective bottles is 0.02 AND 0.02 AND
0.98 AND 0.98 AND 0.98. That is, we’re finding the intersection of five independent events,
so we multiply those probabilities together. In short hand, we find that the probability
of this particular outcome to be 0.02 to the 2 times 0.98 to the three. Let’s look at
another possible way to get two out of five defective bottles: the first and the fourth
bottle are defective. The probability that this happens is 0.02 times 0.98 times 0.98
times 0.02 times 0.98, which is 0.02 to the 2 times 0.98 to the 3. Another way to get
two defective bottles and three non-defective bottles is for bottles three and five to be
defective. The probability of this outcome, just like the others, is 0.02 to the 2 times
0.98 to the 3. We could enumerate all these possibilities, but we’re better than that
– we know some counting functions. How many ways are there to choose 2 defective bottles
out of 5? The answer, of course, is with the combination function from counting. 5 choose
2 is 10. The probability of each of those 10 possible outcomes is 0.02 to the 2 times
0.98 to the 3. It could be the first possible outcome OR the second possible outcome OR
the third, and so on until 10. That is, we’re taking the union of the probabilities of each
possible outcome. That is, we’re summing 0.02 to the 2 times 0.98 to the 3 over and
over again 10 times. We could also write that as 10 times 0.02 to the 2 times 0.98 to the
3. And that is the probability that we find two defective bottles in the five, or the
probability that X equals 2. The probability that any one of those particular outcomes
occurs is 0.000376. When we account for the 10 ways that any of those outcomes could occur,
we get that the probability that X equals 2 is 0.00276.
Let’s write this more generally. For a binomially distributed random variable Y, the probability
that Y takes on value little y is n choose y times p to the y times (1-p) to the n-y.
That is, how many ways are there to choose y successes out of our n trials multiplied
by the probability of a particular outcome: y successes at probability p each, then the
remaining n-y trials are failures at probability 1-p each.
Let’s look at another example. An electronics manufacturer claims that 20% of its power
supply units need service during the warranty period. To investigate this claim, technicians
test 12 units and subject each one to accelerated testing to simulate use during the warranty
period. Let random variable Y measure the number of power supply units that need service
during the warranty period out of 12 units. Let’s find the probability distribution
for this random variable. Recall that the probability distribution lists the realizations
of the random variable, along with the probability of each realization. Let’s find the probability
that X equals 0, that is, the probability that we find no power supply units in 12 that
need service during the warranty period. The probability that Y equals 0 is 12 choose 0
times 0.2 to the 0 times 0.8 to the 12, which is 0.0687. That is, every time we have a set
of 12 power supply units, on average, 6.87% of those sets of 12 will have no units that
require servicing during the warranty period. The probability that Y equals 1, or the probability
that 1 of the 12 units requires service, is 0.2062. Doing this calculation for all other
realizations of the random variable will give us the distribution for Y. Recall again the
two properties of probability: the probability of any realization of the random variable
has to be between 0 and 1, inclusive of 0 and 1, and the sum of the probabilities of
all possible realizations must be 1. Our probability distribution satisfies this requirement, aside
from any round off error with the sum of probabilities.
Let’s take a look at the expected value and variance of a random variable following
a binomial distribution.
For the expected value, let’s first consider the expected value of an individual trial.
If we’re counting successes, then we assign a 1 to the outcome of a Bernoulli trial to
the “success” result of that trial, and a 0 to the failure result. A success occurs
with probability p and a failure occurs with probability 1-p. Therefore, the expected value
of a trial is the realization multiplied by its probability summed across all realizations,
or 1 times p plus 0 times 1-p. The expected value of a trial is p. On average, what’s
the result of a Bernoulli trial? p. We repeat that expectation for n different trials, and
on average, we calculate the expected value of the random variable of a binomial distribution
to be n times p. Back to random variable Y, the number of power supply units requiring
service during the warranty period. p was 0.2, and n was 12. Therefore, the expected
value of the random variable is 2.4 units. That is, when we view 12 power supply units,
we expect on average 2.4 to require service during their warrant period. Note that 2.4
isn’t necessarily a discrete value, as it’s an average number.
For the calculation of variance, remember that variance is the expected value of the
squared difference between the variable and its expected value. Let’s look at one of
the individual Bernoulli trials, the ith trial. The squared difference between the outcome
of the ith trial and its expected value p is p times 1 minus p. When we repeat the trial
n times, we find that the variance for random variable Y is n times p times 1 minus p. The
standard deviation sigma is the square root of the variance. Back to the power supply
example, the variance of random variable Y is 12 times 0.2 times 0.8, or 1.92 power supply
units squared. The standard deviation is 1.39 supply units.
To recap, the binomial distribution is used to describe random variables that count the
number of successes we witness in a number of trials that can have only one of two outcomes,
successes or failures.