Hey guys, this is the first video in a new series that I want to try out on my YouTube channel.
I think I'm going to call it the History of Statistics and I want to make a couple videos
that are a little bit different from the normal tutorial videos that I make in that I want to talk
a little bit about certain topics that I think are interesting. Sometimes there are people or
events that have happened related to statistics and I have learned about them and thought it was
interesting for whatever reason and I think I want to start or make a YouTube series about this. So
my first attempt is going to be with this video, which I think I'm tentatively--we'll see what the
final title is--but I'm tentatively calling it the Rise and Fall of the Iris Dataset. This is a very
famous data set that's often used in statistics or data science or data visualization classes
and I wanted to talk about the history behind this dataset because I think it's interesting.
Without further ado, let me go ahead and get started with the video.
I'm going to begin with some background of where this dataset came from. This is a picture of a man
named Edgar Shannon Anderson. Sometimes he goes in publications by Edgar Anderson or E.S. Anderson,
and he was an American botanist. So he collected this data and first talked about this very famous
Iris flower dataset in a paper published in 1935 and the paper was called The Irises of the Gaspé
Peninsula and it was published in this Bulletin of the American Iris Society. For a couple of these
things that I mention in this video I'll try to put a link to them in the description box below.
I was able to find the paper online and it's a very simple three-page paper. He basically spends
a lot of time just talking about how he has been traveling and collecting measurements on different
irises across the world. This is where the Gaspé Peninsula is, which is where he collected the data
for this paper. This is a picture that I found on Google images. And in this paper he wasn't really
very scientific. It was kind of a very short paper where he spends a lot of time saying that
the countryside in Canada is very pretty and there were lots of flowers, but he doesn't actually
publish the data in this paper and he doesn't really do any statistical analyses either. He
just kind of mentions that he has been collecting data as a botanist for the past 10 years and
specifically he's been collecting data on irises. The reason, he mentions towards the end of the
paper, is that he has been collecting these data, or what he calls dry statistics, because he wants
to test out the theory of evolution or provide data in support of the theory of evolution. So he
collects data for two different species of Irises found in the Gaspé Peninsula and he says that
within a species there is individual variation--so when he collects measurements, for example, on the
size of these flowers, every flower is going to have a slightly different size to it but he
says that he believes that the variation within a species should be less or smaller than the
variation between species. The differences between different species of Irises should be greater than
the individual variation within a species. And so this is why he has been collecting that data.
He says in the paper, "For that reason I have studied such Irises as I could get to see, in as
great detail as possible, measuring iris standard after iris standard and iris fall after iris fall,
sitting squat-legged with record book and ruler in mountain meadows, in cypress swamps, on lake
beaches, and in English parks." It's a very old paper, it's not very scientific, and it actually
doesn't provide the data at all. It kind of is just almost like a journal entry just saying,
"Hey I've been collecting this data." Well, this dataset ended up becoming one of the most popular
datasets in statistics because one year later it was published in a paper by Ronald A. Fisher,
who is basically considered the modern day father of statistics. He publishes this data in a paper a
year later in 1936 called 'The use of multiple measurements in taxonomic problems.' I believe
I read briefly that E.S. Anderson got some kind of money, I think a fellowship or a scholarship
or something, to study in the U.K. and so that's how he ended up meeting Ronald Fisher. And Ronald
Fisher also used to study plants a lot--today he would be known basically as a biologist--and so
that's how they made that connection. And so he mentions, Ronald Fisher in his paper, mentions
that he got the dataset from E.S. Anderson. And the dataset was as follows. There were 50
observations on four measurements of Irises. E.S. Anderson actually only measured two species of
Irises in Canada. He measured Iris Setosa and Iris Veriscolor but in the Ronald Fisher paper there
were three species of Irises. There was a third species that he found observations for and this
was Iris Virginica. Actually when I skimmed this paper, it wasn't really clear to me where he got
this third species from but he does say in the paper that it was separate from the Canada region.
So like I mentioned a little while earlier, Ronald Fisher ended up becoming one of the
most famous statisticians of all time. Any paper that he published would be widely read and widely
cited. Because in this paper he published both an analysis of the data and also just a data
table with the complete dataset, the Iris flower dataset ended up becoming one of the most famous
datasets in statistics. Back then and even today, it's still very widely used. In this next slide,
I just have some sample data visualizations from publications or tutorials I found online
that use the Iris flower dataset. In the original 1936 paper, what Ronald Fisher used the dataset
for was actually to talk about a statistical method called linear discriminant analysis or LDA,
so I have a data visualization of what that looks like. And the Iris flower dataset is
commonly used basically to test different clustering algorithms. It's commonly used
for clustering and other statistical methods as a very simple toy example, or toy dataset.
As further proof that it's a very common dataset, the Iris flower dataset has its own Wikipedia page
and it's also one of the few datasets that is pre-loaded into R. Once you install R, you
don't need an R package to load in the Iris flower dataset. It's one of the pre-packaged R datasets
that you can get with any R installation. I also checked Google Scholar to see how many citations
there were for the Ronald Fisher paper and it says that there are over 19,500 citations for this
paper. Some of those citations might be for linear discriminant analysis but I think the majority of
those are for people using the Iris flower dataset that was published as a data table in that paper.
And I also checked the citations for the original Edgar Anderson paper and there are over a thousand
citations for that paper as well. I think I've shown that this became a very famous dataset.
However, in the last couple of years, there's been a huge fall in the number of people who
use the Iris dataset and a lot of people have been calling publicly for the end of using the
Iris flower dataset. In the next part of the video, I want to talk about why people do not
want others to use that dataset anymore and the main reason is where Ronald Fisher published this
flower dataset. If you see here, this 1936 paper that he wrote about flowers and a statistical
method, he published in a journal called the Annals of Eugenics. Actually that journal does
not exist anymore and today it's called the Annals of Human Genetics. But at the time, Ronald Fisher,
he was a professor but he was not a Professor of Statistics because at the time statistics
departments at universities did not exist, so he actually used to study biology a lot and he was
a professor in a department called the Department of Eugenics. And this is a very controversial part
of the history of Ronald Fisher, which is to say that he was a proponent of eugenics, which if you
don't know what that is you can look it up, but basically it's a very racist way of trying to
say that certain races of people are smarter or better than others. And so today it's completely
considered a pseudoscience and a lot of really bad people in history were proponents or users of
eugenics and they used it to do really bad things. Basically what ended up happening with the Iris
flower dataset is--it obviously has nothing to do with eugenics in the racist sort of way, I mean
it's not even talking about people, it's talking about flowers, but I've noticed a trend being a
couple years ago people have rang the alarm about citing this paper because if you cite this paper,
even though the journal doesn't exist anymore, people would be citing in their own papers
a journal of eugenics. And so since a couple years ago people have been saying, 'Hey it's not
a good idea to cite from this journal because eugenics is not considered a science and it's
very widely denounced nowadays, as it should be.' And so unfortunately the flower dataset that has
again--nothing to do with eugenics--has kind of had this bad reputation placed on it because of
where the dataset comes from and where you would have to cite the dataset coming from
if you want to use it in a paper. This is just a couple examples of screenshots I have seen
because a lot of people--not only do people not use the dataset, but they have been very
vocal about telling other people, "Hey I have stopped using this dataset. You should stop
using the dataset as well.' And so I've seen on statistics twitter a lot of people saying,
'Don't use that dataset anymore because of its association with eugenics.' On Reddit someone
asked, "Is it socially acceptable to use the iris dataset" And actually when I was looking
into research about this topic for this video, I even found a publication that was talking
about some problems with the Iris dataset and I thought it was going to mention this association
with eugenics but I actually found a paper that said there's another problem with the dataset,
which is the measurements. And it said that certain botanists actually disagree that
Iris flowers have sepals--I think that's how you pronounce that word--sepals. So
I read a paper thinking that the problem was going to be about the eugenics, like a lot
of people talk about on twitter, but actually there are other problems that people have with
the dataset as well. Some people argue--and I have no idea about this because I have no
background in botany at all--but some of the measurements some people argue are not even
part of that flower. I don't even know if that's true or not but I found that in a research paper
that was published. These are some examples of the controversy with this flower dataset.
For me, what I remember during this time a couple years ago when people started talking about this,
actually the most famous example that I thought of of people saying that we should not be referencing
Ronald fisher anymore was actually from the American Statistical Association. And so they have
a very famous kind of award and lecture that they called the Ronald A. Fisher Award and Lectureship.
And so this would be awarded annually to a really famous statistician. And it was kind of a big
award and this is a very big organization. And so a couple years ago people were talking about
not using the name Ronald Fisher for the award and this was again tied to his relationship and his
advocacy for eugenics. The last that I remember from this whole discussion and controversy was
that a lot of people wanted to change the name from Ronald Fisher to David Blackwell.
For this video I looked it up again and I found that they had made a change.org petition that had
over 8,000 signatures and David Blackwell, for those of you who don't know, is a very
famous African-American statistician. He was a very great statistician, like a total genius,
I actually want to make a video about him in the History of Statistics series if I continue this,
we'll see how that goes. But he's a very--he has a very interesting history and kind of
biography to him. But I remember that was the last thing I kind of heard about this
news and then when I looked it up for this video, they ended up just removing the name altogether.
They didn't change the name in reference to David Blackwell but they also did remove Ronald Fisher
from the name. I think now it's just called the COPSS Award. Like the ASA was too afraid to name
it after any person unless they get canceled in the future. I think they--I don't know the
reason that they didn't want to change it to David Blackwell but they ended up ignoring the petition
but also removing the reference to Ronald Fisher because he's now kind of a controversial--him and
a lot of other white statisticians who were early founders of statistics are now--people are trying
to move away from naming things after them because they're ties to racist ideology. Now to end this
video, I wanted to talk about an alternative dataset that you can use instead of the Iris
flower dataset. One of the reasons I wanted to make this video was because I actually think it's
kind of an interesting story that an Iris flower dataset collected from Canada that has nothing
to do with race ended up being associated with racism and people saying you can't use this flower
dataset because it's racist. I don't know, I think the history behind that is kind of funny, although
I do agree that we shouldn't be citing from the eugenics journal. Anyway, I've seen different
people say or propose different datasets to use instead of the Iris dataset because honestly
the Iris data set is not that interesting and it's not that intricate. But the number one alternative
that I have seen that people propose is called the Palmer Penguins dataset. These are measurements
from three different species of penguins that you can find in Antarctica and Palmer is a reference
to a research station in Antarctica. I think it's owned by the United States. And so you can
do--again it's three species, just like the three species of Iris flowers. And it's just different
measurements of like their bill and other things, I don't even know all the variables. But this is a
very commonly used dataset that's--people have actually really pushed this idea of using this
penguin dataset instead of the Iris dataset and they have even made these cute little drawings
for the penguins as well. And the good news is if you cite the penguins dataset, the citation has
no reference to eugenics. And so you can do basically the same thing without having to cite
the Annals of Eugenics. And anyway that's all I wanted to discuss for this video. We'll see
if I make another History of Statistics video. But anyway, thank you for watching!
*chiptune music*