So by now we know we’re constantly generating data, using data, or looking at data — hopefully
not as a horrible 3D pie chart.
But it’s not obvious how that data is actually being collected.
Data rarely magically appears on our computers in orderly spreadsheets — we need to put
in some work to get our hands on it.
Like going out in the woods to measure some soil samples, or getting ice cream flavor
preferences from people by just going up and asking them.
Or making a feedback popup that gets them while they're shopping online.
It’s really all hands — or all gears — on deck when it comes to data collection.
Individuals, groups, and automated machines are all helping us gather data.
All these data collection decisions have consequences though.
The way data is collected impacts what kinds of questions we can answer with it.
And the way data is stored and used to answer questions impacts how trustworthy we think
the collectors are.
In this series we’ve been talking a lot about evaluating data, but now it’s time
to get personal.
Who collected the data, how did they do it, and did they do it ethically?
I’m Jessica Pucci and welcome to Study Hall: Data Literacy presented by Arizona State University
and Crash Course.
Being data literate doesn’t just mean we question the results of data analysis.
We also need to question data itself.
And that means both where it came from and how it got here.
If we decide to collect it ourselves, we have first-party data, or primary data, and we
have complete control over how the data is collected.
We can collect exactly what we need without having to sort through extra information.
So suppose we started a blog to share what’s growing in our garden.
We could collect primary data to help us improve by adding a link to a questionnaire and asking
our readers exactly what we want to know, like which seeds they’ve used to get the
tastiest tomatoes.
But in the spirit of collaboration we can share our first-party data and borrow from others.
When we use data collected by another person or group, it becomes second-party data, or
secondary data.
Like, we could ask our blogger friends for their own survey results to get opinions from
a wider variety of readers.
But to use this data effectively we need to ask our friends how it was collected and then
decide if it’ll be useful and fits with our goal — which is plant blogger stardom of course.
And that’s not the only extra data we can get.
Third-party data is collected by an outside company that has no relationship with whoever
they're gathering data on.
This type of data is sometimes bought and sold between companies but could also be open-source
or freely shared by the government.
So we could use third-party data from a tool like Google Analytics to keep track of traffic
to our blog.
But again, we have to be skeptical about this data since we didn’t collect it ourselves.
The more sources of data that are spliced together, the harder it will be to check how
all of it was collected.
And we need to be sure we can actually compare the combined data sets.
We don’t want to have apples and oranges all thrown together, because then our conclusions
might not make sense.
Like, if we find people love reading blogs about how to grow weeds…
Something seems off there.
Whether we’re first, second, or third, all parties have to collect the data somehow.
So just like there’s lots of data collectors, there’s lots of data collection methods.
Anyone can collect observation data, which is basically data we get from what we sense
— you know, like what we see or hear.
Like if we’re at the library looking for a page-turner, we can observe how many people
are in each section.
Observation data is informative but limited by what we have access to.
For more detailed information, there are a lot of other data collections we can opt into.
Like as we’re leaving, we might say “sure we’ll fill out a survey about our library
experience!
The librarian helped us out so much.”
Surveys are lists of questions designed to collect data on a topic or experience from
a particular set of people.
Survey questions can be answered with fixed responses where the survey-maker has given
us answers to choose from, or with open-ended responses where we can add our own ideas.
A mixture of both creates qualitative and quantitative data.
If we have a spare afternoon, we might even be willing to join a focus group and explain
how the library’s new reading challenge affects how frequently we visit the library.
Focus groups are popular in the marketing world and the qualitative data comes from
what participants are discussing.
Interviews are like focus groups and generate qualitative data based on conversation, but
they’re usually one-on-one between a data collector and a participant.
The librarian who helped us choose a book may want to ask us some questions once we’ve
read it.
Survey, focus group, and interview data are all great for getting to know people’s specific
preferences and opinions, but we have to remember we only get this information from volunteers.
If the people who opt in are different from those who opt out of a survey, the survey
results will have self-selection bias, which means our sample doesn’t accurately represent
our population.
If only library patrons who used the free computers fill out our survey, but we don’t
talk to anyone using the library's wifi on their phone, we wouldn’t have the full picture
of how people use the library.
Some data collection even happens without us knowing.
Digital data collection produces large amounts of really personal data, often with us being
none the wiser.
Like every time we scan our library card and check out a book, we actually become a record
in a dataset.
Digital data are often used to find a group’s “revealed preferences,” an idea pioneered
by economist Paul Samuelson in 1938.
He proposed that user preferences are sneakily “revealed” by their history of purchases
— or in the modern era, their history of clicks.
Unlike with surveys, we don’t get to choose to tell a company what our preferences are
— they try to figure them out from our activity.
So yeah technically someone can see our library history.
But it’s unlikely the data analytics team is looking for us specifically.
Even when Netflix called out holiday movie mega-fans on Twitter, the analytics team was
probably looking for interesting patterns, not people.
They likely started by wondering how many times A Christmas Prince had charmed its
way onto screens, sorted the list by individual profiles,
and saw that 53 people were a little more excited about the Christmas-themed royal intrigue
than the average viewer.
Netflix used that data to make decisions.
We now have not one but two Christmas Prince sequels.
Back in the library, check-out records help the library know what books are popular and
how many people are using the library’s resources at a given time.
Digital data is powerful and is a driving force of the big data phenomenon.
And since it’s likely we’re all in at least one dataset, part of being data literate
is just knowing that data is being collected.
But certainly, with great data comes great responsibility.
In the United States, research studies that involve humans have to be approved by an Institutional
Review Board, or IRB.
Europe has the European Network of Research Ethics Committees and there’s also the Forum
for Ethical Review Committees in the Asian and Western Pacific Region, to name a few.
With any project, researchers first make an extensive plan and decide ahead of time what
data to collect — and what data to explicitly not collect — how they’ll store their data
safely, and how they’ll keep up data quality.
A review board evaluates these plans and makes sure they’re ethical, from data collection
to data storage.
They especially look out for any red flags that affect the well-being of any participants.
Let’s see how you might share data about yourself in pursuit of science in the Data Point.
Say you see a flyer for a study taking place at your local hospital that’s looking at
how caffeine and sugar affect concentration.
As an avid coffee drinker, you are intrigued!
Participants will drink coffee, orange juice, or an energy drink and then put together a puzzle.
Health records will be requested and blood sugar levels and heart rate will also be taken
throughout the study.
Woah, woah, woah!
That’s a lot of data to hand over.
But the hospital is required to use some sort of IRB, so the drink you get should not harm
your health.
And the researchers have a plan for safely keeping your data.
Since it’s for science, you decide to sign up for the study and are randomly assigned
to the third group.
So you dutifully chug your energy drink and start puzzling.
While you're in the zone, a researcher takes your blood sugar.
The device makes a weird noise and reads zero, so she shakes her head, tries again with another
device, and gets a much more reasonable number.
As the study finishes, you wonder again about your data.
Your puzzle performance turned out abysmal, and you don’t want that fact broadcasted.
Your secret is safe with the researchers!
Another part of the IRB approval process considers the ethics of data storage.
The safest way to protect your information is to anonymize the data.
This means that a researcher never records your name along with the data that they collect
about you.
You’re just lucky ID number 131313 to them.
Or they might pseudonymize your information so only the researcher knows your identity.
131313 replaces your name on all data files,
but a secret list of names with their associated ID numbers is kept under lock and key or via
password protection.
Phew!
Of course, we know that not all data is collected as part of scientific research.
Data professionals in other fields, like journalism, are expected to have their own standards for
data collection and security.
Like in the UK, the Information Commissioner’s Office provides a data protection guide for journalists.
We’ll talk more in depth on data privacy and security in later episodes, so stay tuned!
Fortunately, standards aren’t limited to storage.
Think about how you couldn’t predict what drink you were given or what group you’d
be in.
To make sure results aren’t hidden by differences in participants, everyone is assigned to different
groups by chance instead of by choice.
This randomization, when possible, is often part of a data collection plan.
Like, would we learn anything about caffeine and concentration if all the coffee lovers
miraculously got the coffee group?
Maybe those participants like coffee because it makes them good at puzzles.
Or they’ve built up a coffee tolerance.
Without randomization we’d never know.
Maintaining those high standards doesn’t stop once we’ve collected the data, either.
The researcher retested your blood sugar because they realized it couldn’t possibly be zero.
We can do the same thing to keep up quality after the data is in a spreadsheet too.
If there’s a record in the dataset, the person must have been alive and well.
So zero or negative numbers for blood sugar can be ignored so that they don’t unfairly
affect the results.
Same thing for extremely high heart rate numbers.
What are you, a hummingbird?
What we’re doing here is data cleaning, or removing data points that are clearly wrong.
If you’re getting the impression much of a data scientist’s time is taken up collecting
data, you’d be right!
It’s a crucial stage that has a huge influence over the conclusion.
So to be critical of a data result, we have to start at the very beginning of the research process.
Today we learned who collects data, how they collect it, and that the ethics of data collection
— like what information we’re taking from people and how we’re storing it —
are important parts of making sure we can trust a data-driven story.
Next time we’ll learn about data literacy from journalists and talk about data in the news.
Thanks for watching Study Hall: Data Literacy, which is produced by Arizona State University
and the Crash Course team at Complexly.
If you liked this video and want to keep learning with us here in Study Hall, be sure to subscribe.
You can learn more about ASU and the videos produced by Crash Course in the links
in the description.
See you next time!