Presented by Dr. Yiftach Nagar
There's a lot of research on
collaborative knowledge co-production systems,
asking how they achieve high quality outputs.
We've learned a lot; we have some recipes
for success
This is particularly interesting
regarding Wikipedia, since Wikipedia's
community has struggled around
such issues as
whether to give credentials to experts
more say or power;
and it has decided not to do so.
Given that stance of the community,
and the research done so far, may lead us
to think that mechanisms and processes
might be enough to distill knowledge
from the wisdom of lay crowds without
expert involvement.
But there are good reasons to think
otherwise; so we wanted to see what
really goes on.
Specifically, we ask:
But we needed some starting point.
Now, if you have someone with a PhD, who
published peer-reviewed research in
a domain
it's relatively safe to consider them as
having expertise in that domain. So we
started to look for such editors.
Although most editors are anonymous,
we were still able to find some editors
who disclose their identities and
credentials,
who have PhD degrees; and we could
cross-check their credentials with their
profiles on university websites and we
further verified that
they indeed published research on the
topic. Our hope was that if we find that
these editors demonstrate
distinguishable editing patterns then
these patterns may also be typical of
other experts
who may have acquired their
expertise elsewhere, outside of academia
At the end of that process we had over
500 editor-article pairs
for which we had established that
the editor was indeed an expert on the
article's topic.
Importantly, though the training was done
on scholarly experts, the classifier
caught experts who are not necessarily
scholars, such as: an avid gamer writing
on video games; or
a Pakistani veteran of the food industry
who writes about Pakistani food, etc.
This confirms our assumption that
regardless of the source of their
expertise,
experts show some
common editing patterns
This has implications for both research
and practice that we're
discussing in the paper and are happy to
discuss further