- [Instructor] What we are going to do in this video
is talk a little bit about experiments in science
and experiments are really the heart
of all scientific progress.
If you think about,
let's just say this represents just baseline knowledge
and then people have hunches in the world
and for a lot of times people say, hey,
I have a hunch that that thing is good for you
or that thing is good for you
but they really had no way of measuring
how confident they were.
They really had no good way of proving it
and even more, because they had no good way of proving it,
it was hard for people to build on top of that knowledge
but with the scientific method and experiments,
people were able to say, hey, we have a hypothesis here
and we were able to do some well-designed experiments
and so we feel pretty good that this is true
and then future people are going to say,
hey, since we feel pretty good that this is true,
maybe we can design experiment to see whether that is true.
Hey, that actually is true and then they can build on that
and we end up having scientific progress
that can accumulate over hundreds of years
and this is really important
that the experiments are well-designed
because in the future and this happens all the time,
we might realize that hey, actually there was a little,
a few assumptions baked in here that weren't accurate
that allowed us to make essentially misleading conclusions.
So our conclusion wasn't quite right there
and then we will have to rebuild from that point
in order to make sure that we are truly making progress.
So the key question is
how do we set up well-designed experiments
and it's a whole field of study
but the whole purpose of this video
is to really give an introduction to it.
So let's just start with a hypothesis.
Let's say that you have a hypothesis
that some pill that is made up of the petals of some flower,
that this pill right over here, it improves,
it improves running, running speed.
It improves running speed if someone were to take it.
So the important thing of any hypothesis,
it has to be testable
and so what you do is you have to think
well, how am I going to test it?
Well, what you can do,
so how are you going to test your hypothesis?
At first, you might say, give the pill,
so give the pill to some runners,
to some runners and test their time,
test their 100 meter time,
test their 100 meter time before the pill,
before the pill and after
and you might say, hey, maybe if, I don't know,
their times improve after, maybe my hypothesis is correct.
Pause this video and see if you feel comfortable
with this test right over here, this experiment.
Well, actually, there's several problems
with this experiment.
How are you selecting these runners
and if you give them the pill and their speed improves,
did it truly improve because of the pill
or did it improve because of some other thing
that they are doing?
Maybe they got new shoes
or maybe their diet improved in some way
or maybe they just had a psychological improvement.
This is often known as the placebo effect.
If people are taking something
that they think will help them, it often will help them
even if that thing is just an empty capsule
or just a sugar tablet.
So how do you avoid these types of errors?
Well, what you could do is you can find runners
and put them into two groups.
So let's say this is one group right over here
and then this is another group and what you would wanna do
is you'd wanna go into the population of people
and you would want to randomly select
whether someone goes into one group or another group.
Why random?
Because if you don't randomly select,
there's a chance that there might be some implicit bias
that you might just happen to be picking people
who maybe their running speed is on an upward trajectory
and they just happen to go into the group
that will eventually get the pill.
So you randomly, randomly put them in those groups
and what you wanna do is you'll have a control group
and you'll have a group that gets your pill
and so this group gets the pill, gets the pill
and now you might be tempted for this group to say,
oh, they don't get a pill and then after a few months of it
and it should be the same amount of time,
you say, hey, did this group's times improve
over the 100 meters?
How did that compare to this group?
But be very careful.
If this group gets the pill and this group gets nothing
then the pill might be providing that placebo effect again
just making people think they're getting something
that's making 'em faster.
It might actually be a self-fulfilling prophecy.
So it's actually important
that you also give these people a pill
although this pill would just look like a pill
so this would be just an empty, empty, empty pill
that looks the same.
Now, there's another idea
when you're designing scientific experiments
that it needs to be double blind.
Let me write this down.
Double blind.
So as you could imagine,
it implies that two things are blind here.
So the first thing that needs to be blind
is the people themselves
should not know which group they're getting put into.
They should not know which pill they are taking
because obviously, if you put someone in this pill,
in this group and you say, hey, you're in the control group,
we're just gonna give you an empty pill,
well, then the placebo effect might not be,
it might not work.
It's also important 'cause it's double blind
that the people who are working with the runners,
the people who are measuring them
so that the researcher is right over here
so I'll draw someone with a clipboard.
So the researchers who are observing these people
and maybe administering the pill
and telling them about the experiment
that they too do not know
which group they are administering it to
because if they did, they might be able to signal somehow.
They might be able to even subconsciously
give a sense of which group folks are in
and so let's say we do all of these things
and so we're getting in the direction
of a well-designed experiment
and we find that the 10 people in this group
versus the 10 people in this group after three months,
these folks had a 5% improvement in running speed
and these people had a 10% improvement in running speed.
Is that enough to conclude that our hypothesis is correct?
Well, you might be tempted, it seems suggestive
but that's where statistics come into order
because there's just some random chance that you got lucky,
that you happened to pick the people that,
your pill does nothing
but you just happened to pick the people
who are going to improve more
and there's a whole field of inferential statistics
when you take a statistics course
that will go in more depth into this
but essentially, what you're gonna do is you're gonna say,
hey, assume that your pill does nothing,
what's the probability of getting this result for 10 people
or what's the probability
of getting this difference in result
and if that probability is very low,
well, you say, hey, that would suggest
that my pill actually does do something.
Now, another important principle of an experiment like this
is it needs to be replicable, replicable
because even though you thought you did a good job,
people might not wanna take your word for it
and it's important in science for people to be skeptical.
When people do experiments, they want to have a result
and that bias might creep in
and so if someone else does an experiment,
you need to say how you did that experiment
so other people can see if they get the same results
because even though you think you randomly selected,
you might only do it with people from a certain country
or under certain weather conditions
or assuming other constraints
and then these people might do it slightly differently
or in a different country or under different constraints
and realize that hey, the explanation for this
maybe was something else.
Another thing to keep in mind
is the larger your samples right over here,
the larger groups that you're able to do this with,
the stronger that the statistics actually become
and I would say not just larger
but the more diverse across genders,
across ethnicities, across geographies.
So the big picture here is all scientific progress
is based on us designing good experiments
and being very rigorous
about how we think about those experiments
and what I've highlighted here is just the beginning
of how we might think about designing those experiments
and as you go into your scientific careers,
look at other people's experiments
and see whether they've done these things
because many times you will find
that it is not as rigorous as it might seem at first.