We select our sample from a population. We want our sample to be representative
of the population, so that after we conduct our well-designed research, what
we learn from the sample can tell us something valuable, by generalizing back
to the population. A representative sample has similar physical or
psychological characteristics as the population. The best way to get a
representative sample is to use a random sample.
However, a random sample does not guarantee a representative sample. A
sample is random when every member of the population has an equal chance of
being selected. The random sample is randomly selected from a larger
population of subjects. Later, we may use random assignment, for example, when we
split our randomly selected sample into two randomly assigned groups. But because
of the way that randomness works, sometimes despite our best efforts, our
randomly selected sample differs from our population in some important way.
Random selection is the best way to get a representative sample, but it does not
guarantee a representative sample. Truly random sampling is difficult to do,
especially when we are studying human beings. Sometimes we don't have the time
or money or availability to do truly random sampling. Even if we were studying
college students at our University, a truly random sample would have to
include all college students across the country, so many times we are left
studying the people whom we have available. They become our participants
because they are convenient, and we call this a convenience sample. The convenient
sample uses participants who are readily available and this is not necessarily a
bad practice, but we should be honest with
ourselves about what we are doing. Before we begin
generalizing, applying the results from our convenience sample to the population,
we should replicate the same study at other universities, with other college
students to see if others find the same results. Most research in the behavioral
sciences is done using convenient samples, and because of this, one joke
goes that psychology is the scientific study of college sophomores. Overstated,
yes, but something to keep in mind when we are using convenience samples.
Convenience samples are more likely to lead to sampling error, and sampling
error occurs when samples are not random or representative. Sampling error happens
when we ask too many of the same people, or we ask the wrong people. The time is
World War 2 in England. In a dimly lit Quonset hut, the Royal Air Force crews
gather having just returned from a bombing run over Germany. The debriefing
begins with a moment of silence for the flight crews who did not return, then the
lieutenant or "leftenant" asks the returning flight crews, from which
direction did the fatal attacks come? The pilots respond to the man "We were
attacked from above and behind." It's unanimous. The leftenant scribbles
this on a piece of paper, hands it to a corporal and instructs, "Take this
information to our departing flight crews, this may save lives. But as the
Corporal is leaving the dimly lit Quonset hut, a hand reaches from the inky
shadows and says stop! No. That information may cost lives so what is
wrong with that information? Stop the video now if you want to ponder or
discuss. This is an example of sampling error.
The lieutenant or leftenant is asking the wrong people. What does he really
want to know? From which direction did the fatal attacks come, but who is he
asking? All we know is that those who survived and returned were attacked from
above and behind. Those who died may have been attacked from a different direction,
which is why those attacks were fatal. Here is another famous illustration of
sampling error. In 1936, the magazine literary digest which had a long track
record of picking presidential winners, mailed 10 million postcards asking
people their choice for president. Of the two million responses, 57% preferred the
Republican Alf Landon. However Landon lost by a wide margin to Democrat
Franklin D Roosevelt. So what went wrong? Again, stop the video if you'd like to
ponder or discuss. The answer this time is sampling error by asking too many of
the same people. The magazine selected names and addresses from phone books and
automobile registration lists, but what year was this? 1936 and what was
happening in America in 1936? The Great Depression. Who could afford cars and
personal telephones? People with money or jobs. Literary digest was asking too many
of the same people. Most poor and formerly middle class people, those
without cars or phones voted for the Democratic Party and FDR, and by the way
the picture of FDR was done by my 15 year-old niece Abby. So let's summarize
what we've learned about populations and samples. Results from the sample should
generalize to the population, that is what is true about the sample should
apply in general to the population. A sample generalizes better when the
sample is more representative of the population. So let's use an example of
cooking stew. You want to know if the stew is ready to serve. Are the carrots
fully cooked? Does it need more salt? Might it contain an ill-cooked bit of
beef or underdone potato? In this example, what is your population? It's the entire
pot of stew. So how large of a sample do you need to determine if the stew is
ready? Do you need to eat an entire bowl? No. You can tell everything you need to
know from a single spoonful, as long as what is true of that spoonful? As long as
the spoonful is representative. The spoonful must contain a carrot, a pea,
some beef, the potato, etc. As long as the elements of the stew are represented in
the spoonful, what we learned from the spoonful can tell us about the entire
kettle. We only need a little taste to know if the stew needs more salt.
Of course, if your spoonful lacks carrots then you cannot answer the question of whether
carrots are fully cooked from your non-representative spoonful. What is the best
way to get the representative sample? The best way to get a representative sample
is from a random sample. That means you stir the pot first so that all of the
elements in the stew have an equal probability of being selected. However,
because a random sample can still be non-representative, we need inferential
statistics to determine how well our sample represents our population.