thank you for watching this presentation
for our paper titled we need a woman in
music exploring Wikipedia's values on
article priority now this is a fairly
lengthy work we're not going to be
able to get to everything in eight
minutes so I really want to leave you
with two main takeaways both
having to do with task routing which is
just the problem of routing tasks to
human experts who can complete them
the first is that task routing is a
multi-stakeholder recommendation problem
emphasis on the multi in
multi-stakeholder and really
this is something that we have to make sure we
keep in mind when we are designing
implementing deploying these kinds of
systems
and the second which builds on the first
in ways that I hope will become apparent
to you as we go through the presentation
is that task routing recommenders on
Wikipedia should incorporate the
composition of existing content and I'll
go into a lot more detail about what
this means
a great place for us to start with
these things is with the motivation for
our work
if you've ever been on Wikipedia's main
page you've probably come across some
version of this message it says welcome
to Wikipedia the free encyclopedia that
anyone can edit 6.4 million articles in
English and it's unsurprising with 6.4
million articles that editors often find
themselves in this situation of of
asking you know what should I edit
where should I start and this is really
a guiding question for our work or
rather helping editors answer this
question is is a guiding question for
our work if you think about this it's
kind of a classic recommender systems
problem you have a bunch of items in
this case 6.4 million of them
and you have a collection of users in
this case Wikipedia editors and you can
use information about those users
like edit history to figure out you know
what are the most relevant of these
items to each user
um you know what would they be
interested in editing based on what
they've edited in the past for example
and sure enough there have been
recommender systems deployed in prior
work that have done just that with a
lot of success and what we want to do
and and what we did is really build on
some of that prior work in a
direction that that we think is is
positive and the way we build on this is
is
if you think about the formulation that
I just gave of the problem it focused a
lot on the editors as the unit that we're
trying to optimize for
you know we ask questions like what
does the editor want what does the
editor like to edit you know what are
the most relevant items for the editor
and we kind of just stopped there
now those were important things to
think about obviously but there are also
other stakeholders that we need to make
sure we're taking into account
readers for example uh you know the
kinds of content that we lead uh editors
towards or guide editors towards is
going to affect what content is
available for readers to consume on
Wikipedia now there's another really
interesting stakeholder which is the
encyclopedia itself
um so if we think about it
Wikipedia as a community as a platform
as a collection of editors has its own
values its own goals interests that
are distinct from those of any
individual editor
and and you know you can see this
come out in a lot of the norms that
Wikipedians appeal to and and hold each
other to
there are other stakeholders but
rather than get into them I just want to
leave you with this takeaway again again
of task routing is a multi-stakeholder
recommendation problem and Wikipedia
isn't an exception to this
um
and we need to make sure that we focus
on more than one stakeholder when
when we're designing task routing
systems
the stakeholder that we chose to
focus on was the encyclopedia because we
found it to be the the least well
studied and well understood
in this context and we did this
through RQ1 which broadly speaking
was what are the encyclopedia's values
for prioritizing articles
the way we got at this is by looking
at a project called vital articles where
Wikipedians basically compiled lists of
50 000 of the highest priority articles
for English Wikipedia
and the way they compiled these lists
is through discussion and argument and
manually you know working together to
build consensus and decide you know
what should be there and what should be
removed
through you know years-long
process of iteration
and what's really great about this is
that all these discussions
are publicly available on the the
platform and so we decided to just
qualitatively analyze them and figure
out you know what are the criteria that
Wikipedians use to say hey this is
really high priority for English
Wikipedia to have at high quality and
conversely like you know this is very
low priority for for Wikipedia to to
have at high quality
what we came up with was this list of
eight criteria which all of them are
very interesting in isolation but really
what I want to focus on is the
existence of global criteria
um
and I'll start with kind of what the
takeaway is of global criteria and then
I'll get to what the criteria are so the
takeaway is just that and this is rather
intuitive if you want to figure out
how high priority a given article is you
can't just look at properties of that
article itself you also have to look at
what the state of the encyclopedia is
so for example what does the
encyclopedia need
what is the encyclopedia missing
um
these are questions that have to factor
into those kinds of decisions and and
those are really captured in these three
criteria of balance non-redundancy
and completeness and an example of
balance that we found that came up again
and again in these vital articles
discussions was balance along gender
dimensions and and that's something that
you know we definitely need to be
careful to incorporate in task routing
recommender systems in in guiding
editors towards tasks that are going to
be productive to forward the
encyclopedia's values
um
and so if we look at you know examples
of the gender composition of of
Wikipedia we see that amongst highest
quality featured articles
there's a huge disparity in
biographies of men and biographies of
women and so the takeaway from this is
that really
we don't want to be designing
recommender systems that are going to
exacerbate this
you know even if we have a situation
where you know the the biographies of
men are just more relevant to each
individual editor in the aggregate we
want to make sure that we are actually
counteracting this trend and not
exacerbating it
and that leads again to this high
level takeaway of task routing
recommenders on Wikipedia should
incorporate the composition of existing
content and we do a lot more exploration
in the paper of this dimension and
and other dimensions of balance
non-redundancy and and completeness
um
so with that if you have any
questions please do feel free to reach
out otherwise I hope you will check out
the paper which builds a lot on
some of the things that I discussed
today and and really adds a lot more
interesting content and and follow-up
there
thank you very much