Hey everyone, thank you very much for inviting me to be here and organizing this socially distant CHI.
My name is Jenny Fan, and I’d like to present our paper, ”Digital Juries: a civics-oriented approach to platform governance”, on behalf of myself and my co-author, Professor Amy Zhang.
The widespread reach of social media platforms magnifies the negative effects of toxic content online.
This includes mis and disinformation, dog-whistling, hate speech, and other forms of harassment.
It’s often considered ”borderline” and requires highly localized context and awareness of sociocultural norms to judge and understand.
Society is increasingly concerned, rightfully, about the scale and ease by which online speech can incite violence, radicalize viewers, and even impact elections.
This topic has been covered at length in recent years.
With a few notable examples of community-driven moderation, many of these “industrial scale” platforms use a mix of algorithmic moderation and human content moderators to make these decisions.
However, a drawback of regulating speech by algorithms, contract workers, and even external boards is that platforms lack meaningful ways to invite democratic participation from its user base.
This leads us to our primary motivation for our study. Regulating speech is a highly subjective, normative task that requires much more user participation than what is currently afforded by commercial content moderation.
Without a democratically legitimate process that all users agree with, this cycle of conflict and disagreement around the outcomes of each moderation decision will inevitably continue.
In the language of Tom Tyler’s procedural justice framework, we need a system imbued with procedural justice, where users believe the process is just regardless of whether they agree with the outcome or not.
In democratic governance, one of the most important non-electoral means of civic participation is jury duty.
Juries are a method of group decision-making that serves 2 roles: one to determine a group’s preference, and another to determine social norms and their relationship to law. 15 00:01:56,00 --> 00:02:05:00 The value of juries as a norm-enforcing institution for content moderation could be quite valuable, but has thus far only had limited applications online. 16 00:02:05:00 --> 00:02:13,00 If used at all, they’re often a transitional step towards eventual automation or used for mass polling, rather than supporting authentic deliberation for participatory governance.
Thus, to explore the potential for Digital Juries, our paper describes these three contributions:
First is our 5-stage model, which outlines the design space of a digital jury process.
Second is our prototype hosted on digitaljuries.com, which we built ourselves and is flexible to adapt to a variety of cases.
Finally, we used the prototype to test and gather empirical evidence about whether digital juries are perceived as more democratically legitimate than the alternatives.
To design this system, we first investigated a number of self-governing communities to study how they regulate behavior in online spaces.
Many platforms today maintain a customer-to-merchant relationship with their user base, but communities that are more self-governing have something closer to a citizen-sovereign relationship, as coined by Lawrence Lessig. 23 00:03:03,30 --> 00:03:10:00 In these types of communities, users have many more mechanisms to contribute to the governance of their spaces. 24 00:03:10:00 --> 00:03:18,00 Taking inspiration from Elinor Ostrom’s work on governing the commons, a few examples to note are Wikipedia’s famously decentralized processes,
distributed platforms like Reddit, Mastodon, or Minecraft,
and the game League of Legends, which featured a Tribunal system were players could actively participate in judging in-game violations committed by their peers.
While these communities have a number of community-supported governance tools, they all stop short of using juries in a civic way.
The LoL Tribunal successfully operated for three years but was not used by the platform as a civic exercise.
Riot Games eventually replaced the Tribunal with an automated system in 2014, citing reasons of slowness and low efficiency.
The various benefits and drawbacks to these precedents motivated us to define a model for a digital jury process, which clearly decomposes the complex group decision-making dynamics of a constitutional jury process into one that could be more easily evaluated online.
It consists of 5 stages: first is the Jury selection process, which includes the pool jurors are drawn from and how a jury is composed.
Next is the onboarding to the system, which also includes training and how they are incentivized – whether it’s by payment, reputation, volunteering, civic duty, etc.
Third is the actual case itself, which features the content in question and also the format of how evidence is presented.
Fourth is the consensus forming process, such as how jurors collaborate to form a decision. This could have varying amounts of synchronicity and happen in a variety of mediums, such as text chat, video chat, message boards, etc.
This category also includes decision rules, such as majority or unanimous rule.
Last is the enforcement of the jury’s decision, including what actions the community will take to punish and how that record is preserved.
Our study conditions focus on 2 versions that vary Stage 4, the Consensus stage, as this medium can be uniquely modified for online juries. However, there are many interesting alternative experiments to explore here, which we’ll mention in our Future work.
Many of the model parameters would change depending on which online community we studied, so we chose to instead focus on the jury process in the abstract and built our own system. We designed a study to answer our three main research questions –
First, how would users perceive the democratic legitimacy of juries, as opposed to the status quo?
Second, how would users perceive the legitimacy of a jury consensus process around blind voting, versus deliberation?
And finally, would users prefer these jury decisions be enforced directly, or passed onto platforms as recommendations?
The prototype is hosted at digitaljuries.com and is designed to be a flexible system that shows cases to juries of up to 6 participants.
As a brief orientation to the interface, users are shown cases with relevant background context on the violation up top.
In the bottom panel, users can give inputs to evaluate the case.
Finally, the Decision Output panel on the top right shows the verdicts from the other jurors in your jury.
To evaluate a case, users were asked to rate the toxicity of each case from 0 to 10, as well as decide what punishment action to take on the content as well as the user.
We built 3 different versions of this jury interface:
one that replicates the status quo, which requires no user inputs
one that is more “scaleable” and decides consensus by a blind vote within the jury
and one that is more “immersive” that involves a 4 minute chat-based deliberation.
Our jury system is abstracted away from the context of any particular social media platform, which gave us more flexibility and removed any existing biases towards a particular platform.
We designed a study using Mechanical Turk, since we needed participants who could feasibly be online at the same time. Our resulting demographics ended up with a similar skew to that of the Mturk population online at the time we recruited.
It was a within-subjects experiment with 3 rounds, so all users could compare all 3 conditions of the jury, each with a different case. Both the cases and jury conditions were counterbalanced. After each round, users filled out a survey and free response questions.
The cases were written by the authors and all involved potentially borderline judgments. This is an example of the Status Quo condition with the Pepe case.
This is an example of the Scaleable, or Blind Voting, condition with the Momo urban legend case.
This is an example of the Immersive, or deliberating condition, with the Christchurch shooter case.
In the post-round surveys, we asked users to rate their perceptions of democratic legitimacy on 6 dimensions, based on aspects of procedural justice and Jonathan Haidt’s Moral Foundations framework.
The 2 jury conditions were rated higher than the status quo on every aspect, except for being an effective way to protect users from toxic content.
There was no significant difference between the jury conditions, except on the aspect of caring about personal preferences, where the deliberating version rated higher.
Users were also asked to rate their satisfaction with each condition, and directionally, the users rated immersive condition as the highest satisfaction. The status quo was the least preferred by 55% of participants.
The survey differences between the 2 jury conditions were not as clear, but a couple interesting themes emerged from open-ended responses.
Those supporting deliberation valued the diverse viewpoints and group collaboration offered by the medium.
On the other hand, supporters of voting mentioned the greater efficiency and protection against groupthink.
This individualistic theme of valuing ones’ own opinion but distrusting others, either other jurors or other platform administrators, emerged a number of times in open responses.
The voting juries tended to assign higher toxicity ratings with a larger spread of results than when they deliberated, though these differences were not significant.
Interestingly, deliberating juries chose to warn users more frequently.
while voting juries chose to instead ban users at nearly twice the rate of deliberating juries. So, the scaleable condition was harsher.
When it came to whether jury decisions would be enforced or only recommended, we saw no statistical difference in preference. Both options ended up with a mean of 3.34 and mode of 3 on the 5-point Likert scale.
Despite the lack of a clear preference, participants did express strong views for and against both sides. Pro-enforcement users were concerned about a lack of accountability to the public, and 12 jurors expressed cynicism regarding platform motives.
However, pro-recommendation users were concerned about potential juror bias. Interestingly, both sides called for greater transparency in publishing jury decisions, whether it was to audit the platform or to audit other users.
From the open-ended fields in our study, we noticed a paradox in platform governance: the distrust of public discernment was at odds with the competing distrust for platform discernment.
The question of who was more trustworthy, when it came to other humans or automated systems, presented 2 competing mental models of trustworthiness.
Similarly, the inclusion of other jurors’ opinions in jury systems raised 2 competing views on fairness: a pluralistic view where hearing other people's perspectives was a benefit, while the more individualistic camp saw it as a potential vulnerability for influencing others.
These themes flowed over to feasibility considerations for a jury system as well.
Many jurors expressed concerns about biased or bad actors exploiting the juror selection process, and especially expressed doubts about the incentives of paid jurors.
However, in our post-study survey, 82% of jurors expressed interest in participating in such a system if paid. Here, reframing jury service as an empowering civic duty rather than compensated micro-task could change this discussion.
Of course, when dealing with such toxic material, a jury process would have to carefully follow best practices from criminal court proceedings and content moderation policy in order to protect the jurors from harm. At a minimum, the exposure would be more limited than professional moderators.
Lastly, the challenges of finding an entire jury of distributed users to convene synchronously during the deliberation stage was difficult. Future systems could look into automated scheduling or jury assignment based on whoever is online.
For our future work, we’re particularly drawn to 2 perspectives:
First will be understanding the impact of jury bias and diversity on online juries, which will be vital to protect minority voices, and also gauge the accuracy and effectiveness of juries.
Diverse juries have been shown to decrease polarization, and studies here could target juror recruitment and study their effects on content moderation decisions.
Additionally, deepening our understanding of jury localization would involve broadening the understanding of how juries are composed, both across rounds and across time.
The locality could be defined by geography, nationality, or other types of group memberships.
Finally, the flexibility to convene any number of juries online would enable localized interpretation of policies in a way never thought possible with current industrial-scale social media platforms.
In conclusion, our paper’s focus on digital juries as a civics-oriented approach toward platform governance opens up a rich area of design opportunities for online communities.
Our study’s findings are summarized here, and the prototype system we built is available at digitaljuries.com.
With that, we’d like to conclude by thanking the Harvard Masters in Design Engineering program for making this thesis research possible, and the organizers of CHI 2020 for giving us a chance to present our work. Thank you very much!