Peer review is not a simple thing because humans do it and so whenever there’s humans
there will be bias introduced, relationships get complicated with regard to peer review.
We have a fantasy that peer review will be totally anonymous on both ends – you won’t
know who the person you’re reviewing is and they won’t know who you are and it will
all be blind.
But in reality people figure out in peer review who is who and biases do get introduced.
But that said, the idea of peer review is a really important one and we can sort of
approach it and that’s the idea that we have people who are qualified judging each
other’s work.
It allows us to essentially crowd source knowledge and to have an opportunity where blind spots
are picked off, where errors are picked up and where we can make work better.
So it’s essentially a way to crowd source knowledge.
I want to point out by the way that peer review is historically really interesting.
It came out of the enlightenment period.
So this was the period when thinkers were beginning to really appreciate the idea that
humans together could know more.
And what’s fascinating is that democracy and science grew up together and they both
use peer review.
So science uses peer review because scientific ideas are put forth and then scientists are
qualified to do so, judge that work.
And in democracy peer review is used in things like voting systems so when we do voting that’s
a peer review form.
When we do judging of criminal or non-criminal acts in courts that’s a form of peer review
when we have a jury.
And so it’s not a coincidence because what was happening was the thinkers of the enlightenment
were beginning to figure out that more people looking at a problem could to get you better
knowledge.
Before the enlightenment the concept was knowledge came from above, it came from the church,
from the state, from God, it came from an external higher authority.
But the real revelation of the enlightenment was the idea that people could do this themselves
they didn’t have to rely on the church, the state, God, an external authority, they
could to do it themselves.
And so they began to have the idea that they would reject the king and they would essentially
reject the teachings of the church and they would reject the state being run by the king;
that they would take back control of knowledge.
And that was true in democracy and in science.
So, it’s no coincidence that a lot of the founding fathers were science geeks.
They were thinking about crowdsourcing.
It is what we call crowdsourcing.
There are more checks and balances on bad knowledge going forward so there’s accountability
at some level.
When you’re doing pier review the editor at least knows who you are.
When you’re doing voting theoretically you’re not allowed to vote more than once.
When you’re on a jury you have got a judge keeping track of whether or not information
should be admissible in court whether it’s fair to admit it in court.
The internet is crowdsourcing gone wild.
It has no limits on it and so you can have things like bots like things until it actually
is noticed by real human beings, you can have situations where something looks incredibly
real but it’s not real and it will take off much faster than we can stop it.
So the internet really is a beautiful thing in many ways.
It allows people to find each other who never could have found each other before, for example,
people with very unusual medical conditions can find each other, people with very unusual
interests can find each other.
The problem is that you have a situation where there are no checks and balances and so you
get a phenomenon whereby things that are not real can go forward.
But there are some places on the Internet where there are checks and balances.
So Wikipedia is a great example actually.
Wikipedia actually has people who function as editors and they will talk to each other,
fight with each other and all the discussions get externalized.
That allows a level of accountability that much of the Internet doesn’t have.
It’s also the case that Wikipedia paid editors can actually stop people from editing in some
circumstances or stop people from messing with pages.
So there are places on the Internet that have been born of crowdsourcing but that do have
some checks and balances built in.