hey sv dustin welcome back to smarter
every day unfortunately now is the time
for the video about disinformation on
reddit the front page of the internet
it's been documented by both the
european union and the united states of
america that russia iran china are all
spreading disinformation online about
the coronavirus pandemic which to be
clear can kill you if you don't know the
right things about the pandemic you can
die
so why would they do that this is the
fourth video in a series on
disinformation we've already covered
youtube twitter and facebook okay reddit
does not want disinformation on the
platform in fact they're doing a ton to
try to fight against it but the question
is how do you fight disinformation
without giving up what makes reddit so
powerful a place to freely exchange
ideas with other people anonymously
today we're going to talk about this
tension and more importantly how it
affects you and i as reddit users to
learn about this let's go get smarter
every day and talk to the experts we're
going to speak to chris slow the chief
technology officer at reddit and then
we're going to go to rene de resta at
the stanford internet observatory we'll
talk on the phone with john at the
oxford internet institute and then
jeremy blackburn at the idrama lab it's
a group of scientists and academics that
use math and science to try to detect
this stuff all right first off let's go
to actual reddit headquarters something
i've always wanted to do we're going to
meet chris slo chief technology officer
and get this conversation started i
guess let me start here yeah who
controls reddit somebody controls reddit
that's a good answer that's a good
answer but i mean reddit is
it's the front page of the internet like
you go to reddit and like whatever's on
that front page it influences culture
for that day because i think the actual
answer is like
everybody who uses reddit controls
reddit in some in some way you want
people to upvote or downvote yeah it's a
function of you know if the community
likes that content and so yeah the next
level is the moderator so every every
community has its own set of moderators
okay and they enforce their own set of
rules and so they get a chance to the
tone of their community so they can take
down anything they want directly they
can just remove it um
and they can set the rules to be as
strict or as loose as they want to be
the underlying important unit is not the
content or the user but the communities
that are built on the platform is
coordinated in authentic behavior a
problem on reddit um i mean it's
definitely a problem we constantly work
to prevent
um i would say like
there will always be examples of things
that hit the front page that potentially
shouldn't be there that's kind of like a
nature of the platform right because
you're dealing with a bunch of people
right and so um you know
you know extreme example would be yeah
click bait is a thing like titles that
are suggestive that will get you to
click through or like you know um
but what we try to do is there's two
parts of it one is
users have an opportunity to reconsider
their vote after seeing the content um
secondarily
and you know they have the download
available to them as a way to kind of
like degrade any any um inauthentic like
behavior that happens the other side of
it is like you know we put a lot of work
over the last decade and change into
maintaining a certain like sanctity of
the vote um you know the joke i usually
use is that um
we count every vote but that doesn't
mean that every vote counts
right and so we have to be able to come
up with
um you know based on based on behaviors
knowing that there's some adversary on
the far side who's trying to basically
game our system
have a set of
somewhat opaque ways to hide like with
what counts and what doesn't but at the
same time make sure that we're not you
know throwing the baby out with the
bathwater right like we do want to have
things that are popular
manifest themselves as popular
but
during that very early formative time
that really first couple of votes is
what's really critical right that's
where there's a lot of scrutiny around
making sure that you know there's not
like a game going on where you know it's
it's me and my 50 000 clones that are
all coming from the same computer and
the same ip address or accounting
towards that photo you can think of it
as like this kind of a tiered system for
how we deal with the way content appears
on reddit um at least for like the like
the enforcement side of things or like
the removal side of things the the last
line of defense is us like us as a
company okay um you know the admins um
as we are referred to on the platform um
we our job is to enforce
like site-wide policy violations like
you know copyright violations anything
that violates our our content policy you
know harassment spam all that great fun
stuff
our appearing in a place is like the
last line of defense it's like the it's
like you're sending the national guard
to clean up like a giant mess okay so
like you're saying when reddit employees
get involved yeah i mean that that's a
rare thing is what you're saying that's
a relatively rare thing just a personal
observation reddit seems to be way more
hands-off when it comes to this stuff
than other social media platforms but
it's also way more community focused
rene de resta has a ton of experience
studying this information online so i
asked her what she thought about this
community driven approach first there's
an interesting idea there which is that
the community should decide what kind of
content it wants to tolerate right and
um and so you do see things like i think
there's a subreddit where um you can
only post a picture of a cat and if you
post a picture of a dog like you know
it's deleted and banned and nobody goes
in there and screams that they're being
censored because they couldn't post the
dog into the cat subreddit so it's
interesting in light of like the
moderation challenges faced by like
twitter and facebook where there's this
expectation that
one moderation standard is is uh is fit
for the whole community i do think
reddit is an interesting experiment
seeing how that much more hands-on
moderation kind of activity works and
then also
when the community reports these weird
accounts coming in
there is a little bit more of like
you know you as a member of that
community have a better sense of like
where that uncanny valley is where like
the content's not quite right the um
the person typing the account like you
know sorry the person typing the comment
gets it just a little bit wrong they
don't understand the community yeah
exactly and and so there is like i i do
believe that reddit has a unique um that
community moderated kind of point of
view
um for detecting anything from
disinformation to harassment at the same
time we do see these accounts get
through right and so the question is
also what's their top level strategy
for
managing and detecting and being
proactive recognizing that their
platform is a target for state-sponsored
actors is it enough to rely on community
mods or is there also something in place
to
have a
top-level view that's doing a little bit
more of the investigative type work that
facebook and twitter are doing so when
speaking to renee and chris we talked
about upvote manipulation down vote
manipulation brigading there's all kinds
of manipulation that can take place on
reddit and those discussions are pretty
long so i'll leave them over on the
second channel at some point but for now
there was one little nugget that fell
out of the conversation with chris from
reddit that i think other redditors
might appreciate one of the one of those
um size and variant things on reddit has
been the uh the uploaded dominant ratio
consistent over 14 years i don't know
what this even means seven to one
there's been seven upvotes consistently
to every down vote cast on the site
that's interesting i don't really i
don't get it but it's a it's a thing
that's always happened so i'm sitting
here with jeremy blackburn he's from
binghamton university in new york the
top public university in new york and he
runs this thing or at least participates
in this thing called the eye drama lab
which i don't really understand what you
are could you please explain the eye
drama lab for me yeah sure so i drama
lab we're a group of international
scientists and
we specialize in getting a under having
a large scale a very wide and high level
view holistic view of the internet as a
whole so
we
don't just look at twitter or just read
it or just facebook we look at it all
and we focus really on understanding the
connections between them the hidden
connections between them and other
chunks of the web
this is really important we think we're
you know we're pretty good at it
it's our niche and uh we do our job
there got it so let me just ask you this
straight up then
is there some type of large-scale
coordinated inauthentic activity on
reddit is that a thing
yes absolutely
really the internet has a long history
of people
pretending to be something they're not
uh and
the internet kind of enables that so no
i don't i don't don't want to say that
everybody you see that's acting a little
bit weird is is some kind of bad actor
there are people that are new to
communities they have to learn the rules
you know maybe they are interested they
just haven't learned the culture yet but
it's not crazy to be
on the lookout for this stuff it does
happen there are active
campaigns to to abuse social media and
influence opinion so what are you seeing
uh we're seeing that there's a lot of
content going on right now with
especially with the coronavirus
unfortunately we see that a lot of this
stuff appears to be effective still um
it there's not we haven't noticed yet at
least any kind of news particularly new
strategies that are being used we're
still seeing kind of the same basic
strategies that were two three years ago
because they still work i guess one
thing i didn't get from the reddit
interview is they were kind of top chris
was kind of top level he didn't really
dive down and say oh yeah user surf
ninja 385 did this do you have like a
specific example of a time when
somebody's trying to manipulate reddit
yeah there's a so reddits actually put
out their stuff um there's a reddit.com
wiki suspicious accounts and on that
site uh what they did and actually i
give them a lot of credit for doing this
um following the 2016 election reddit
acknowledged 944 trolls that have been
active on their platform and it did a
very detailed transparency report in
early 2017 or maybe late 2017 but they
did a really detailed transparency
report where they actually go through
and you can see that the user rubinger
for example has a karma of uh 99
400 it looks like and reddit for a while
um had left these accounts up um and so
you could actually kind of click in
there and you could see
what they were who they you know where
they were posting to and that was where
you could start to see that they were
posting to some of the funny meme sites
some of the black community subreddits
um some of the far right community
subreddits and so to their credit reddit
did actually make this visible to the
public they kept them up there for quite
some time which is an interesting choice
it is because
facebook took it all down even though
they flagged the accounts when i first
heard that reddit was leaving up access
to both posts and comments from known
russian agents who were trying to
manipulate us i thought that was a
really odd decision on reddit's part but
when i went to the public link reddit
security provided of all the known troll
accounts i realized that this was
actually valuable it lets you study the
context of the comments and the post to
understand how these disinformation
campaigns work at the time this
particular troll operation was underway
the strategy seemed to be an attempt to
use the hot button issues of the day in
order to rile up americans on both sides
of any argument all the while making
sure to cycle in a good helping of low
effort cute animal repost to try to
throw reddit security and other users
off the scent at the time of these posts
in 2015 and 2016 they clearly focused on
racism and police issues they would go
from subreddit to subreddit hoping that
it would explode and make everyone fight
and create as many casualties as
possible the comments were interesting
they did one of two things they were
either camouflaged trying to convince
other redditors that they were real
people or they were inflammatory hoping
to nudge people towards hating each
other although there were a few outlier
accounts that seem to be focused more on
comments if you look at the karma scores
of the vast majority of these accounts
the post karma was on average 40 times
larger than the comment karma so
inflammatory post seemed to have been
the weapon of choice that being said
even though the comments seem to appear
less influential let's take a moment and
try to analyze them to understand the
cumulative effect that they did have on
the conversation i'm on the phone with
john from the oxford internet institute
and he's been researching mint's
information on reddit and he's going to
talk to us a little bit about what he
sees in the data so you have a report in
robo-trolling which is the magazine nato
strategic communication center makes
quarterly about online disinformation
you've got a graph down here that i was
hoping you could explain to us uh sure
i'll give it a try so
this was looking at what happens to the
reddit conversations after uh some
manipulations so as you've got a reddit
thread sort of
ticking along over time um you think of
this left to right in each of those
graphs you know another comment another
comment another comment and then in the
middle on the red dotted line we have
when a a fake account a russian ira
troll account has made a comment they've
injected something into this into this
thread
and then what we have is the baseline
which is tracking along at zero
is essentially a control thread so
another thread that hasn't been
manipulated
and then what we see in each of the
three lines is how the manipulated
conversation diverged so what happened
where was the change cognitive
complexity what does that mean so a
higher
cognitive complexity score an acceptance
that multiple viewpoints can be true
even if they're in opposition whereas a
low complexity score is singular very
single-minded i'm right and there's
nothing else that could possibly be true
okay an identity attack i assume that
means like people start attacking each
other based on who they are like gender
religion things of that nature yeah
exactly attacking somebody because of
their their identity so a higher
toxicity score suggests that it is more
aggressive more confrontational and it's
more likely to lead the recipient of the
message to leave the conversation and
this is what i think is really
interesting is that we see in all three
of these measures the conversations did
change
so they became on the left-hand graph
they became
less diverse in the number of opinions
they became more more polarized single
viewpoints
and in the middle there was a sort of a
short spike in the number of identity
attacks uh although this did return to
baseline afterwards and then on the
right we see a sort of sustained rise in
the level of toxicity in the
conversation so so this is just one
comment that was able to make the entire
conversation diverge is that what i'm
seeing here uh yes so this is just at
the one the one comment level so each
one individual comment did create a
measurable change in the nature of the
conversation so the magnitude of this
change is small but measurable but if
you scale this up to thousands of
comments over the whole platform then
suddenly that starts to have a bigger
impact wow it's right like this is data
this is pretty clear if you can
make a a conversation into a two-sided
thing instead of a three-sided or
four-sided whoever sided thing it
becomes much easier to control it right
if there's two very clearly opposing
points of views black and white
now it's easier to control the narrative
because people get locked into one of
those points of views or the other using
this same data provided by reddit jeremy
and the idrama lab team were able to
create a report with fascinating
implications this graph shows weekly
troll activity plotted over many years
this yellow line is what we're
interested in because it's russian troll
activity on reddit what you can see here
is that there was a bunch of activity by
those accounts in the second half of
2015 and then the activity seems to go
dormant and then it creeps back up and
spikes in the fall of 2016. the initial
activity in 2015 seems to be karma
farming they would free boot content
from other places on the internet and
post it for karma on reddit this post
for example looks awfully familiar and
so does that pistol by the way hmm
weird after this long period of what is
thought to be karma farming to give the
username more credibility the troll
activity seems to slow down for a while
and then when the moment is right in
this case right before the 2016 election
a percentage of these accounts surge
into action and try to influence society
so it's pretty clear what's happening
they're creating the accounts and then
they're grooming them to become
effective social media weapons which can
then be deployed at the exact moment
when they might be the most effective
think about where we are right now
global pandemic the early stages we're
coming up on an election everybody's
kind of tense at the moment if you had
these social media weapons these
accounts parked
when would you deploy them what is a
home run
for a person or entity that would try to
run a misinformation campaign on reddit
what would be their goal i think that
the most achievable goal is just to
cause chaos to cause
uh polarization and to to keep people
from going towards a common goal to
deliberately drive people apart so you
see this type of
uh behavior elsewhere on twitter it's
been very well explained that there are
there were state-sponsored actors taking
both sides of the same argument right
they didn't have particular gold they
just wanted to cause problems jeremy
showed me something that made me realize
we're only scratching the surface with
this series we're looking at individual
platforms here right twitter facebook
youtube reddit they all have their own
vulnerabilities he showed me data and
explained that these campaigns are
coordinated attacks that span the whole
internet and it appears that reddit is
being used as a way to scale
disinformation campaigns from smaller
places on the internet up to more
mainstream outlets military is aware
that this is a big deal they know it's a
big deal and
nobody really has a map of the
battlefield a full map of the
battlefield yes facebook has their you
know their map of you know facebook land
and twitter has their map of twitter
land but um uh you know nobody knows
what's going on in all on other parts of
the internet and it's hard to win a war
without a map i don't know if it's ever
been done
it's hard to win a war without a map
that's pretty good dude
so what would you say to the people on
reddit right now
that they're using the platform and
they're starting to question what they
see you know they're seeing maybe why is
this kind of stuff at the top and if
they do identify some type of
inauthentic or coordinated behavior like
what what do we need to tell them to do
report it definitely no click the report
button that's what the report button is
for okay um uh we look at we look at
reports we look at
anything that you send to us we do we do
try to look at the more people who are
reporting stuff and identifying things
that look a little off
that's the thing that really helps us to
find it and localize it so that's a
signal that you see that's a signal we
can use and of course like you know the
obvious question is like what do we do
about people who abuse the report button
of course people will be the report
button of course we know how to deal
with that um so i think you know the
broader the base of people who are
reporting stuff who tell who think
things look wrong
the more more signal we have it goes
back to the only way to scale up users
with more users like you know we we can
either have a gigantic police force that
um and like a bunch of ai equipped
cameras that watches everybody at all
times what could possibly go wrong or we
can let the you know let the
neighborhood watch take over i would
rather live in a society that has a
neighborhood watch rather than a bunch
of cameras on every street pool so chris
is talking about this neighborhood watch
but i was very surprised when i look at
the data for myself and i realized that
reddit is very proactive when it comes
to taking down bad actors on the
platform in the fourth quarter of last
year there were about five and a half
million reports of potential content
manipulation on reddit by users in that
same period reddit removed almost 31
million pieces of content and sanctioned
almost 2 million accounts for content
manipulation to be clear that means
reddit has taken down roughly six times
more content and accounts than have been
reported if reddit overreacts uh if they
you know get too uh authoritarian and
authoritarian if you will that can push
people
um even further away from what we might
consider mainstream or normalcy uh type
of stuff because then they feel
persecuted and stuff like that so reddit
has a difficult job
you know maybe they could do a better
job but uh you know i don't envy that
job either
you're pretty happy with what they're
doing
i don't know if i'm happy with it but uh
i don't i also don't know if i have a
better a better easily implementable
answer there are definitely worse things
they could be doing so clearly reddit
admins are proactively working to
decrease the influence of bad actors on
the platform i find this interesting
because when i started this study i was
under the impression that reddit was too
hands-off but i'm starting to understand
that it's more of a fine line that they
have to walk also and this is more
personal when i first started studying
this stuff and i started seeing these
disinformation campaigns around me
i think something bad happened to me my
reaction to this new reality was flat
wrong i started to irrationally see
trolls everywhere oh that person feels
strongly and disagrees with me must be a
troll oh look a pot shot at me in the
comments that's a troll ignore that
person and it started to feed this us
versus them narrative from a different
angle the the angle where i think anyone
that disagrees with me must be a troll
at the risk of confusing you i'm going
to point this out
the troll's first play is to make you
hate your online brother
the troll's second play is to make you
think your online brother is a troll if
you go through the comment database you
can see trolls meta joking about the
existence of trolls so what would you
say to the normal user of reddit the the
person who goes there to find funny
memes
uh the person that you know their kids
are sending them links from reddit what
do you say to that person well i think
we don't want to create a feeling of
paranoia right you don't want people to
think like everybody i talk to on the
internet is a pseudonymous you know
troll operating out of
some other country but at the same time
i think there's like the need for a
healthy skepticism if somebody's posting
stuff that
really makes you feel riled up or really
makes you feel
strongly
uh
you know kind of take the extra two
seconds to
to do the check you know where did this
come from and why do i feel compelled to
share it knowing that trolls appeal to
emotion is very important because they
are real so here's what i'm going to do
what if i were to assume everyone was
real hear me out on this not like a
username to interact with or a comment
to interact with but like actual people
and i make that primary and i just go
forward on the internet as if they're
real when i'm more kind and more loving
it seems to be more toxic to trolls
think back to john's graphs if i see
someone trying to reduce the cognitive
complexity of a conversation i'm going
to add nuance and try to expand the
conversation
that's toxic to trolls if i start to see
identity attacks in a thread i'm going
to call it out in a non-toxic way with
kindness and love which is also toxic to
trolls if i de-escalate the rhetoric and
try to make it less aggressive and less
confrontational
that's toxic to trolls and plus this is
how i want to interact with people in
real life reddit has a very difficult
job they've got admins mods they have
other tools in place to try to thwart
the bad guys but for me
the real battle is with me every
interaction i have on reddit is the one
i can do something about and i want to
be a good guy and i want to upvote
people that are doing the same so if i
see you out there decreasing the
toxicity or trying to expand the
conversation
you're going to have my upvote i want to
say thank you thank you for watching
this big series that i did on
misinformation we had youtube twitter
facebook and now reddit there's even a
cyber warfare video before that this was
a challenge and i want to say thank you
to the patrons for letting me mentally
have the freedom to do this people that
support at patreon.com smartereveryday
that frees me up i don't feel like i'm
tied to the algorithm i can just explore
the things that i genuinely want to
explore
and i thought this was an important
topic if you feel like this series has
earned your subscription i would greatly
appreciate that there's even a little
notification bell if you click that
you'll be notified when i upload but if
not no big deal if you want to discuss
this we'll be doing it over at r
i'll leave links down below for all the
references that we talked about and
other than that i am destin you're
getting smarter every day thank you for
the support have a good one bye