Hi, I'm Casey. I'm a professor and I specialize
in technology ethics. And I would love to get to a world where technology is
more awesome and also hurts people less. So let's talk about zoom bombing.
Actually let's talk about awful people who do awful things with technology and
how tech companies should be thinking about
that. I hope that's enough to get you to watch to the end.
Now before I get started, this video contains references to
awful people doing pretty awful things. So if you're not having a kind of day
where you can deal with that, then maybe bookmark this and watch it later.
So if you are somehow lucky enough to have not heard about this new scourge on
humanity that is zoom bombing, let me fill you in. Remember how in the
pre-garbage fire world of 2020 people mostly used video conferencing
for work and remote collaboration. This is
certainly nothing that a five-year-old or your great-grandmother would ever had
to have worried about or used. However, now in our
socially distanced post-pandemic world, well, all we do is zoom.
[Music: "Rumpshaker" - All we wanna do is zoom-a-zoom-zoom-zoom...]
I'm sure that analysts have had a field day with why zoom
took off as opposed to other video conferencing platforms
but let's just say that it did. Suddenly it was
April 2020 and we're in a whole new world and
the people who used zoom before were just wearing business suits over their
pajamas, but now it's just people wearing their pajamas
like everyone else in the world. Kindergarten classrooms,
virtual happy hours, city council meetings,
weddings, funerals, and basically everything else. But it's
keeping us socially connected! It's keeping us sane! And then awful
people had to go and ruin it. because that's what awful people do.
[Newscaster] A problematic trend known as zoom bombing has emerged.
In recent weeks, zoom has been plagued by targeted
trolling tactics. You know if you build it the trolls will come.
Where intruders hijack video calls many of these attacks
involving hate speech and racist images
[Interviewee] They started shouting racial epithets. They shouted the
n-word. [Newscaster] A group of zoom bombers crashed the meeting,
unleashing profanities and racial slurs. [Zoombomber] All y'all have coronavirus. Every single one of you.
[Interviewee] We were being violated and degraded in our own homes and I hope no one else experiences that.
And it goes on and on. Now we can talk about why this happened.
Zoom had some particularly poor privacy protocols.
Arguably some not great design decisions when it comes to this kind of thing.
And for whatever reason some people think this
is funny. Some people even think it's funny enough to put their own exploits
on youtube.
[audio from a zoombombing]
Isn't it funny to make people
uncomfortable. But it is not my goal in this video to
try to analyze why awful people are awful.
It's just the fact that they are. But how did we get here? Because this
is not new. Let's go back to the early days of the internet.
Before zoom bombs and deep fakes and doxing and social media harassment
campaigns or really social media as we know it at
all. Imagine online interaction as simple as a text-based
MUD, which stands for multi-user dungeon, but basically think like
world of warcraft but in text. Like zork or colossal cave adventure but with
lots of other people. i actually played one of these on a
dial-up bulletin board when I was in middle school. It was called Legend of
the Red Dragon. The murky forest stands before you. What
do you do? Well you look for something to kill of
course. Or you could have a conversation at a bar with other patrons. Or you could
just take out the role-playing element entirely and go straight to a hundred
percent socialization like LambdaMOO. So LambdaMOO
was established in 1990 and it was one of the first really
popular online communities. And when i say really popular i mean like 10,000
people but at the time that was a pretty big deal. This seems really peaceful
right? Like how could people possibly be awful
here in this text-based utopia?
Okay here's where I'm going to talk about some awful people being awful.
If you don't want to hear discussion of rape,
even the virtual kind, then I will give you a time stamp and you can skip this
part. In 1993 journalist Julian Dibell
wrote an article for The Village Voice titled
A Rape in Cyberspace. This article actually eventually led to an entire
book called My Tiny Life where Julian talked about his
adventures and experiences in LambdaMOO. He also went on to do a lot of work
around like virtual economies and Second Life and gold farming and this sort of
thing. Also now he's a lawyer which i totally get because my
fascination with the early days of the internet was also why I went to law
school. Leep in mind as you listen to this that
this was a 100% text based environment The first line of the
article is: They say he raped them that night. So in
LambdaMOO someone who was going by the name Mr. Bungle created a
sub program that allowed you to subscribe actions to other characters
within LambdaMOO. By falsely attributing actions to other characters in the
community it was essentially like they could make your body do
whatever they wanted. And you can imagine just how awful this got. When this
happened in LambdaMOO it obviously caused a huge amount of
discussion in the community about the boundaries between virtual reality
and real life and how the community should
be governed. And in response to this specific event
LambdaMOO changed. They added features that allowed for things like
booting people out of the system and also changed the way that they governed
the community. And this was in reaction to someone doing something awful.
And as Julian Dibbell reveals at the end of the article,
it turns out that Mr. Bungle wasn't even just a person. Mr. Bungle was essentially
communal property of a floor of a college dorm. So the young
man who was on the keyboard the night of this rape was not alone. He was being
egged on and given suggestions by a group of
other people. When I first read My Tiny Life in 2004
during grad school take one I was well aware of
the awful things that people could do online. But
even looking back now that seems like such
simpler more innocent times than we have now.
Now it just seems like, hey here's a new technology,
how is it going to be used to harass people? So here's a more
recent example. Last year some research that was led by my PhD advisee Aaron
Jiang resulted in a paper about moderation
practices on Discord. The difference between
Discord and other types of similar online communities
is that voice chat is a huge part of it. And it turns out that when you talk to
the people who do the moderating in an online community,
you find out about all of the bad things that people do there,
and all of the new ways that they can figure out how to break rules.
You know how when you see a warning label that says something like
don't hold the chainsaw on the wrong end, and you think someone must have been
clueless enough to hold the chainsaw on the wrong end
or we would need a warning label. Similarly when you're in an online
community or like a subreddit and you see a rule
that says do not spam this community with
pornography, you know that that rule probably exists
because someone spammed that community with pornography.
For example when we talked to discord moderators they described a behavior
that they called, and i'm not a fan of this term,
ear raping. This is disruptive, invasive noise that is intended to hurt
people's ears or their sensibilities. Like literal screaming or playing
pornography very loudly. And similarly kind of like zoom bombing
there are raids on Discord which apparently people also
think is funny to share on youtube. So one of our takeaways of
the study is how hard it can be to moderate or
regulate or govern new technologies because you might not even know yet how
people are going to be awful or how people are going to figure out to break
the rules. And this brings us back to the question of why are awful
people awful. I'm not going to try to answer that but
the important thing is we know that they're going to be. We know
that new technologies are going to be misused.
We just know that. It's not 1990 anymore. For example google drive. We know that
kids are using google drive to cyberbully each other
because social media is being blocked at their school so why not just create a
burn book and pass it around? Also turns out there is no way to stop
anyone, an abusive ex partner for example, from
sharing google drive files or folders with you
and if you think about it for a second you can imagine how that kind of thing
might be misused. Google drive, even that can be used to
harass people. Okay so now we're back to zoom.
And here's why all this awful people will be awful stuff is relevant.
Because again it's been 30 years almost since Mr Bungle. We should be expecting
this by now. So as zoom was criticized for poor
privacy protocols the CEO did a number of interviews.
[Newscaster] Eric Yuan is the founder and CEO of zoom. He's got a zoom background
up right there. Eric, what happened in the past few weeks?
[Eric Yuan] Now our service was built to serve business and a lot of enterprise
customers. However during this COVID 19 crisis we moved too fast.
Our intention to serve the end the users of the k-12 schools,
however, we have we had some missteps. And he also gave this quote to the new
york times: We were focusing on business enterprise
customers. However we should have thought about
what if some end user started using zoom for non-business events,
maybe for family gatherings for online weddings.
The risks, the misuse, we never thought about that.
So my question is why? We'll think about it later
is actually pretty common in technology design, which is not super surprising
considering that sometimes you want to move fast and break things.
There's even a term for this: technical debt.
Sure there's going to be bugs in the software but we'll fix them later.
So technical debt is this implied future cost of having to refactor the code to
fix these things once you know what the bugs are.
And it's gonna suck to fix it later, probably more than it would suck to fix
it now, but hey at least we can get it out. Gotta
hit those release dates. So my PhD advisee Natalie Garrett and i
have been thinking about ethical debt. Like sure this new technology might
cause harm. It might have some huge social
implications but we don't know what they are now we'll
fix them once we know what happens. Of course
especially in this case by the time the debt collector comes
it's too late. The harm has already been done. The harm has to be done for you to
know what it is. And those students and funeral goers
they can't unsee or unhear whatever the zoom bombers threw at them.
So what do we do about it? Well one thing is we should be thinking
about the risks the misuse
during the design process. For example user experience designers or really any
kind of technologist often create user personas to imagine
ways that people might use their technology. These user personas
should involve personas like user stalking their ex
user who thinks it's funny to traumatize vulnerable people
racist user, sexist user, user who thinks it's funny to show
everyone their genitals. And if you're not doing this i think
you're missing a really important design step.
And that's even if you're creating something as innocuous
as a productivity software because boy are awful people creative. And if the
way that you're thinking instead is, well sure maybe people will misuse
our tech but we'll figure it out after they do it,
then start keeping an accounting book of your
ethical debt. And if all i can do to convince you
otherwise is to point out that you could have a PR disaster then that's fine, I'll
point that out yeah you could have a PR disaster.
Like this was not a good look for zoom. Another thing is
you know who might be really good at imagining ways that technology might be
used to harass people? People who get harassed a lot. Which
means women and LGBTQ people and people of color.
In other words the exact people who are underrepresented in the tech industry.
This is one of the reasons why diversity in tech is important.
Bbecause also these harms tend to disproportionately affect
marginalized and vulnerable people and if all you're thinking about
is most of your users, then the same people keep getting
harmed over and over and over again. Because the problem with the debt
is that it has to be paid back with interest.
And maybe for technical debt bug fixes aren't such a big deal,
but for ethical debt if harms propagate then it's everyone who's paying the
price. And one of the things that this means is that we can't always trust tech
companies to protect us. This is why it's really important for
everyone to have a good sense of how to protect your own privacy online for
example. And if you're using zoom on a regular
basis, make sure that you understand the
features, that you're setting passwords and using waiting rooms and doing
everything else that you can to prevent zoom bombing. I'll link to some resources
down in the description that make some suggestions about what
you can do to protect yourself. Right now some of the research that my
students and I are doing in my lab is around how to help
technology designers speculate about future harms. I really
hope that this is something that increasingly becomes an integral part of
the design process so that hopefully we can avoid some of these
kinds of blow-ups in the future. Though on an optimistic note maybe we
can also just convince people to not be so awful.
I would like to think that that's possible. But until it is
be safe out there. If you're new to my channel i do mostly post videos about
academic advice so far. However I do plan to keep making some of these video essay
style videos about my research including tech ethics
and online communities and other awesome things that I like to do. So if
you liked this video give it a thumbs up and subscribe because I would love to
have more people find these. Check out the
description below for links and references to things that I just
discussed. And I'm Casey. Thanks for watching.