When I collect data, I either want to collect data from the whole population, or just from

a sample of the population.

ìPopulationî doesn't necessarily mean every person in the entire world.

If Iím interested in ìWhat kind of food should I sell in the tuckshop?î then the

population is everyone who has access to the tuckshopóstaff, students, parentsóbut not,

for example, the people who go past on the road outside the school.

Theyíre not part of the population.

The population is everyone ìof interestî.

But the problem with asking everyone at the school about the tuckshop is: A, Iím going

get way more data than I can deal withóand of course everyoneís got a different opinion;

and B, how am I actually going to get hold of every single person at the school?

Thatís quite hard; doable, but hard.

But imagine instead you want to know what kind of food to sell in a new shop at the

Hyperdome.

Now whoís the population?

Thatís not merely hard, itís probably impossible.

Why not just ask a smaller group, a sample that represents the population?

Of course, that raises the interesting question of how do I choose the sample so that it does

in fact represent the whole population?

How do I avoid a bias in my results because I asked the wrong people?

For example, if I only ask the year five students what they want in the tuckshop, Iím going

to end up with a menu full of sugar and who knows what else, it probably wonít suit even

the much older or younger students, and it certainly wonít satisfy the adults at the

school or the studentsí parents!

How do I choose a fair, representative sample?

Weíll come back to these questions in a moment.

There are some times when we do talk to the whole population, and thatís called a ìcensusî.

For example, the Australian Government conducts a census every five years.

They ask every single person in Australia to fill out a form wherever theyíre staying

that night.

And everyone in Australia should be on one of those forms somewhere.

Thatís a census: when you ask every person in the population.

But if on the other hand you only ask some people, who youíre going to assume represent

everyone, thatís a ìsurveyî.

Thatís the connection between those words.

If youíre asking just a sample of people, thatís a survey.

If youíre asking everyone in the population, thatís a census.

So letís suppose youíre going to conduct a survey.

You need to choose suitable questions, to get the kind of data you want; and you need

to choose a suitable sample who actually represent the greater population.

First of all, you can ask basically two kinds of questions.

Closed questions are ones where the surveyor decides what answers are allowed.

For example, have a look at this survey on the right, which I scanned from a political

leaflet I received in my mailbox.

The first question there is a closed question: You have to number 4 boxes from 1 to 4.

You have to choose four issues from that list.

You canít make your own selection of issues that matter to you.

Well, of course, you can.

You can write anything you like on the pageóor of course you can just pick up the phone and

call the politician, have a real conversation about whatever you like!

But the survey itself is designed to get you to choose from the set responses only.

This is a closed question.

But the next question on the survey is open.

You can respond in any way you like.

For the person who has to analyse the results of the survey, closed questions are much simpler:

You just tally up the responses to each option and straight away see where the priorities

lie.

Open questions, on the other hand, are much harder: You have to sift through the different

responses and abstract out the common issues, get a feel for the general kinds of responses

people are making.

Much more work to analyse.

Iím not saying that closed questions are always bad and open questions always better.

Sometimes if you ask an open question, people wonít know how to respond.

(Imagine a letter-box survey with the first question just asking ìWhat four issues concern

you most?î and some blank lines to write on.)

You have to design the survey carefully to make sure you get valid, representative answers

without biasing the responses people make.

Which brings us to bias.

It is possible to bias the results of your survey by the kinds of questions you ask;

this is questionnaire bias.

For example:

ìThe kangaroo is a beloved Australian icon and in some areas numbers are decreasing alarmingly.

You wouldnít be in favour of anyone wanting to kill them, would you?î

As opposed to: ìIn some parts of Australia, there are so many kangaroos that they cause

road accidents and prevent planes from landing on airstrips.

They can also create unsanitary conditions in parks where children play.

You wouldnít object to their numbers being culled, would you?î

See how the wording of the question (and the way itís asked) makes you feel inclined to

answer in a particular way?

Or try this one: ìLying to the tax office is a crime.

Did you cheat on your last tax return?î Like who is going to say ìYesî?

Thatís questionnaire bias.

You can also bias the results by asking the wrong people (or not asking the right people).

This is sampling bias: Choosing a sample whose views do not accurately represent the population.

For example, remember when we asked only the year five students what to put in the tuckshop?

Our choice of sample biased the results, because the views of year five students donít accurately

represent the views of everyone else at the school.

Or suppose you want to find out whether thereís community support for a new football field

to be built.

You could try asking people who turn up at a classical music concert one evening, but

you would probably get biased results because theyíre not necessarily people who would

want or use a football field.

You have to ask yourself: Do these peopleís views accurately represent the views of the

population at large?

That political survey has a severe case of sampling bias.

Who actually responds to such things?

I didnít; I just scanned it to use as an example in my maths lessons.

Do I care about these issues?

Yes, of course I do.

And I vote.

But the thing about post-back and call-this-number-now surveys is that only certain kinds of people

respond to them.

And theyíre often the people with the most extreme views, either extreme in nature or

extreme in how strongly-held they are.

The vast majority of people simply donít bother to reply.

Will you really get views that fairly represent the population?

Finally, interpretation bias.

This works two ways.

First, the questions you ask may be misinterpreted by the people youíre surveying.

For example, you may be using words that have different meanings to certain kinds of peopleólike

asking mathematicians about whether something is ìfairî.

Or take the second box on that political survey, ìTackling local crime in our communityî.

Whatís crime, vandalism or shoplifting?

Or break & enters?

Or the hoons doing burnouts at 2am?

Or (to return to that earlier question) tax cheats?

Theyíre all crimes.

Never mind what ìlocalî or ìcommunityî mean.

But it can also work the other way.

Just because 80% of respondents say they care about local crime, doesnít mean you know

what they want you to do about it.

You might interpret that to mean that police numbers should be increased, or that laws

need to be tightened up and magistrates be more strictóor alternatively that the laws

are cancelled, so that those things are no longer crimes!

(Well, that sounds radical, but itís a perfectly valid interpretation of the results of that

survey.)

You see how these issues are all closely interlinked.

What you want is a survey that fairly represents the views of the whole population and whose

results have an unambiguous, valid interpretation.