Chris Anderson: What worries you right now?
You've been very open about lots of issues on Twitter.
What would be your top worry
about where things are right now?
Jack Dorsey: Right now, the health of the conversation.
So, our purpose is to serve the public conversation,
and we have seen a number of attacks on it.
We've seen abuse, we've seen harassment,
we've seen manipulation,
automation, human coordination, misinformation.
So these are all dynamics that we were not expecting
13 years ago when we were starting the company.
But we do now see them at scale,
and what worries me most is just our ability to address it
in a systemic way that is scalable,
that has a rigorous understanding of how we're taking action,
a transparent understanding of how we're taking action
and a rigorous appeals process for when we're wrong,
because we will be wrong.
Whitney Pennington Rodgers: I'm really glad to hear
that that's something that concerns you,
because I think there's been a lot written about people
who feel they've been abused and harassed on Twitter,
and I think no one more so than women and women of color
and black women.
And there's been data that's come out --
Amnesty International put out a report a few months ago
where they showed that a subset of active black female Twitter users
were receiving, on average, one in 10 of their tweets
were some form of harassment.
And so when you think about health for the community on Twitter,
I'm interested to hear, "health for everyone,"
but specifically: How are you looking to make Twitter a safe space
for that subset, for women, for women of color and black women?
JD: Yeah.
So it's a pretty terrible situation
when you're coming to a service
that, ideally, you want to learn something about the world,
and you spend the majority of your time reporting abuse, receiving abuse,
receiving harassment.
So what we're looking most deeply at is just the incentives
that the platform naturally provides and the service provides.
Right now, the dynamic of the system makes it super-easy to harass
and to abuse others through the service,
and unfortunately, the majority of our system in the past
worked entirely based on people reporting harassment and abuse.
So about midway last year, we decided that we were going to apply
a lot more machine learning, a lot more deep learning to the problem,
and try to be a lot more proactive around where abuse is happening,
so that we can take the burden off the victim completely.
And we've made some progress recently.
About 38 percent of abusive tweets are now proactively identified
by machine learning algorithms
so that people don't actually have to report them.
But those that are identified are still reviewed by humans,
so we do not take down content or accounts without a human actually reviewing it.
But that was from zero percent just a year ago.
So that meant, at that zero percent,
every single person who received abuse had to actually report it,
which was a lot of work for them, a lot of work for us
and just ultimately unfair.
The other thing that we're doing is making sure that we, as a company,
have representation of all the communities that we're trying to serve.
We can't build a business that is successful
unless we have a diversity of perspective inside of our walls
that actually feel these issues every single day.
And that's not just with the team that's doing the work,
it's also within our leadership as well.
So we need to continue to build empathy for what people are experiencing
and give them better tools to act on it
and also give our customers a much better and easier approach
to handle some of the things that they're seeing.
So a lot of what we're doing is around technology,
but we're also looking at the incentives on the service:
What does Twitter incentivize you to do when you first open it up?
And in the past,
it's incented a lot of outrage, it's incented a lot of mob behavior,
it's incented a lot of group harassment.
And we have to look a lot deeper at some of the fundamentals
of what the service is doing to make the bigger shifts.
We can make a bunch of small shifts around technology, as I just described,
but ultimately, we have to look deeply at the dynamics in the network itself,
and that's what we're doing.
CA: But what's your sense --
what is the kind of thing that you might be able to change
that would actually fundamentally shift behavior?
JD: Well, one of the things --
we started the service with this concept of following an account,
as an example,
and I don't believe that's why people actually come to Twitter.
I believe Twitter is best as an interest-based network.
People come with a particular interest.
They have to do a ton of work to find and follow the related accounts
around those interests.
What we could do instead is allow you to follow an interest,
follow a hashtag, follow a trend,
follow a community,
which gives us the opportunity to show all of the accounts,
all the topics, all the moments, all the hashtags
that are associated with that particular topic and interest,
which really opens up the perspective that you see.
But that is a huge fundamental shift
to bias the entire network away from just an account bias
towards a topics and interest bias.
CA: Because isn't it the case
that one reason why you have so much content on there
is a result of putting millions of people around the world
in this kind of gladiatorial contest with each other
for followers, for attention?
Like, from the point of view of people who just read Twitter,
that's not an issue,
but for the people who actually create it, everyone's out there saying,
"You know, I wish I had a few more 'likes,' followers, retweets."
And so they're constantly experimenting,
trying to find the path to do that.
And what we've all discovered is that the number one path to do that
is to be some form of provocative,
obnoxious, eloquently obnoxious,
like, eloquent insults are a dream on Twitter,
where you rapidly pile up --
and it becomes this self-fueling process of driving outrage.
How do you defuse that?
JD: Yeah, I mean, I think you're spot on,
but that goes back to the incentives.
Like, one of the choices we made in the early days was
we had this number that showed how many people follow you.
We decided that number should be big and bold,
and anything that's on the page that's big and bold has importance,
and those are the things that you want to drive.
Was that the right decision at the time?
Probably not.
If I had to start the service again,
I would not emphasize the follower count as much.
I would not emphasize the "like" count as much.
I don't think I would even create "like" in the first place,
because it doesn't actually push
what we believe now to be the most important thing,
which is healthy contribution back to the network
and conversation to the network,
participation within conversation,
learning something from the conversation.
Those are not things that we thought of 13 years ago,
and we believe are extremely important right now.
So we have to look at how we display the follower count,
how we display retweet count,
how we display "likes,"
and just ask the deep question:
Is this really the number that we want people to drive up?
Is this the thing that, when you open Twitter,
you see, "That's the thing I need to increase?"
And I don't believe that's the case right now.
(Applause)
WPR: I think we should look at some of the tweets
that are coming in from the audience as well.
CA: Let's see what you guys are asking.
I mean, this is -- generally, one of the amazing things about Twitter
is how you can use it for crowd wisdom,
you know, that more knowledge, more questions, more points of view
than you can imagine,
and sometimes, many of them are really healthy.
WPR: I think one I saw that passed already quickly down here,
"What's Twitter's plan to combat foreign meddling in the 2020 US election?"
I think that's something that's an issue we're seeing
on the internet in general,
that we have a lot of malicious automated activity happening.
And on Twitter, for example, in fact, we have some work
that's come from our friends at Zignal Labs,
and maybe we can even see that to give us an example
of what exactly I'm talking about,
where you have these bots, if you will,
or coordinated automated malicious account activity,
that is being used to influence things like elections.
And in this example we have from Zignal which they've shared with us
using the data that they have from Twitter,
you actually see that in this case,
white represents the humans -- human accounts, each dot is an account.
The pinker it is,
the more automated the activity is.
And you can see how you have a few humans interacting with bots.
In this case, it's related to the election in Israel
and spreading misinformation about Benny Gantz,
and as we know, in the end, that was an election
that Netanyahu won by a slim margin,
and that may have been in some case influenced by this.
And when you think about that happening on Twitter,
what are the things that you're doing, specifically,
to ensure you don't have misinformation like this spreading in this way,
influencing people in ways that could affect democracy?
JD: Just to back up a bit,
we asked ourselves a question:
Can we actually measure the health of a conversation,
and what does that mean?
And in the same way that you have indicators
and we have indicators as humans in terms of are we healthy or not,
such as temperature, the flushness of your face,
we believe that we could find the indicators of conversational health.
And we worked with a lab called Cortico at MIT
to propose four starter indicators
that we believe we could ultimately measure on the system.
And the first one is what we're calling shared attention.
It's a measure of how much of the conversation is attentive
on the same topic versus disparate.
The second one is called shared reality,
and this is what percentage of the conversation
shares the same facts --
not whether those facts are truthful or not,
but are we sharing the same facts as we converse?
The third is receptivity:
How much of the conversation is receptive or civil
or the inverse, toxic?
And then the fourth is variety of perspective.
So, are we seeing filter bubbles or echo chambers,
or are we actually getting a variety of opinions
within the conversation?
And implicit in all four of these is the understanding that,
as they increase, the conversation gets healthier and healthier.
So our first step is to see if we can measure these online,
which we believe we can.
We have the most momentum around receptivity.
We have a toxicity score, a toxicity model, on our system
that can actually measure whether you are likely to walk away
from a conversation that you're having on Twitter
because you feel it's toxic,
with some pretty high degree.
We're working to measure the rest,
and the next step is,
as we build up solutions,
to watch how these measurements trend over time
and continue to experiment.
And our goal is to make sure that these are balanced,
because if you increase one, you might decrease another.
If you increase variety of perspective,
you might actually decrease shared reality.
CA: Just picking up on some of the questions flooding in here.
JD: Constant questioning.
CA: A lot of people are puzzled why,
like, how hard is it to get rid of Nazis from Twitter?
JD: (Laughs)
So we have policies around violent extremist groups,
and the majority of our work and our terms of service
works on conduct, not content.
So we're actually looking for conduct.
Conduct being using the service
to repeatedly or episodically harass someone,
using hateful imagery
that might be associated with the KKK
or the American Nazi Party.
Those are all things that we act on immediately.
We're in a situation right now where that term is used fairly loosely,
and we just cannot take any one mention of that word
accusing someone else
as a factual indication that they should be removed from the platform.
So a lot of our models are based around, number one:
Is this account associated with a violent extremist group?
And if so, we can take action.
And we have done so on the KKK and the American Nazi Party and others.
And number two: Are they using imagery or conduct
that would associate them as such as well?
CA: How many people do you have working on content moderation
to look at this?
JD: It varies.
We want to be flexible on this,
because we want to make sure that we're, number one,
building algorithms instead of just hiring massive amounts of people,
because we need to make sure that this is scalable,
and there are no amount of people that can actually scale this.
So this is why we've done so much work around proactive detection of abuse
that humans can then review.
We want to have a situation
where algorithms are constantly scouring every single tweet
and bringing the most interesting ones to the top
so that humans can bring their judgment to whether we should take action or not,
based on our terms of service.
WPR: But there's not an amount of people that are scalable,
but how many people do you currently have monitoring these accounts,
and how do you figure out what's enough?
JD: They're completely flexible.
Sometimes we associate folks with spam.
Sometimes we associate folks with abuse and harassment.
We're going to make sure that we have flexibility in our people
so that we can direct them at what is most needed.
Sometimes, the elections.
We've had a string of elections in Mexico, one coming up in India,
obviously, the election last year, the midterm election,
so we just want to be flexible with our resources.
So when people --
just as an example, if you go to our current terms of service
and you bring the page up,
and you're wondering about abuse and harassment that you just received
and whether it was against our terms of service to report it,
the first thing you see when you open that page
is around intellectual property protection.
You scroll down and you get to abuse, harassment
and everything else that you might be experiencing.
So I don't know how that happened over the company's history,
but we put that above the thing that people want
the most information on and to actually act on.
And just our ordering shows the world what we believed was important.
So we're changing all that.
We're ordering it the right way,
but we're also simplifying the rules so that they're human-readable
so that people can actually understand themselves
when something is against our terms and when something is not.
And then we're making --
again, our big focus is on removing the burden of work from the victims.
So that means push more towards technology,
rather than humans doing the work --
that means the humans receiving the abuse
and also the humans having to review that work.
So we want to make sure
that we're not just encouraging more work
around something that's super, super negative,
and we want to have a good balance between the technology
and where humans can actually be creative,
which is the judgment of the rules,
and not just all the mechanical stuff of finding and reporting them.
So that's how we think about it.
CA: I'm curious to dig in more about what you said.
I mean, I love that you said you are looking for ways
to re-tweak the fundamental design of the system
to discourage some of the reactive behavior, and perhaps --
to use Tristan Harris-type language --
engage people's more reflective thinking.
How far advanced is that?
What would alternatives to that "like" button be?
JD: Well, first and foremost,
my personal goal with the service is that I believe fundamentally
that public conversation is critical.
There are existential problems facing the world
that are facing the entire world, not any one particular nation-state,
that global public conversation benefits.
And that is one of the unique dynamics of Twitter,
that it is completely open,
it is completely public,
it is completely fluid,
and anyone can see any other conversation and participate in it.
So there are conversations like climate change.
There are conversations like the displacement in the work
through artificial intelligence.
There are conversations like economic disparity.
No matter what any one nation-state does,
they will not be able to solve the problem alone.
It takes coordination around the world,
and that's where I think Twitter can play a part.
The second thing is that Twitter, right now, when you go to it,
you don't necessarily walk away feeling like you learned something.
Some people do.
Some people have a very, very rich network,
a very rich community that they learn from every single day.
But it takes a lot of work and a lot of time to build up to that.
So we want to get people to those topics and those interests
much, much faster
and make sure that they're finding something that,
no matter how much time they spend on Twitter --
and I don't want to maximize the time on Twitter,
I want to maximize what they actually take away from it
and what they learn from it, and --
CA: Well, do you, though?
Because that's the core question that a lot of people want to know.
Surely, Jack, you're constrained, to a huge extent,
by the fact that you're a public company,
you've got investors pressing on you,
the number one way you make your money is from advertising --
that depends on user engagement.
Are you willing to sacrifice user time, if need be,
to go for a more reflective conversation?
JD: Yeah; more relevance means less time on the service,
and that's perfectly fine,
because we want to make sure that, like, you're coming to Twitter,
and you see something immediately that you learn from and that you push.
We can still serve an ad against that.
That doesn't mean you need to spend any more time to see more.
The second thing we're looking at --
CA: But just -- on that goal, daily active usage,
if you're measuring that, that doesn't necessarily mean things
that people value every day.
It may well mean
things that people are drawn to like a moth to the flame, every day.
We are addicted, because we see something that pisses us off,
so we go in and add fuel to the fire,
and the daily active usage goes up,
and there's more ad revenue there,
but we all get angrier with each other.
How do you define ...
"Daily active usage" seems like a really dangerous term to be optimizing.
(Applause)
JD: Taken alone, it is,
but you didn't let me finish the other metric,
which is, we're watching for conversations
and conversation chains.
So we want to incentivize healthy contribution back to the network,
and what we believe that is is actually participating in conversation
that is healthy,
as defined by those four indicators I articulated earlier.
So you can't just optimize around one metric.
You have to balance and look constantly
at what is actually going to create a healthy contribution to the network
and a healthy experience for people.
Ultimately, we want to get to a metric
where people can tell us, "Hey, I learned something from Twitter,
and I'm walking away with something valuable."
That is our goal ultimately over time,
but that's going to take some time.
CA: You come over to many, I think to me, as this enigma.
This is possibly unfair, but I woke up the other night
with this picture of how I found I was thinking about you and the situation,
that we're on this great voyage with you on this ship called the "Twittanic" --
(Laughter)
and there are people on board in steerage
who are expressing discomfort,
and you, unlike many other captains,
are saying, "Well, tell me, talk to me, listen to me, I want to hear."
And they talk to you, and they say, "We're worried about the iceberg ahead."
And you go, "You know, that is a powerful point,
and our ship, frankly, hasn't been built properly
for steering as well as it might."
And we say, "Please do something."
And you go to the bridge,
and we're waiting,
and we look, and then you're showing this extraordinary calm,
but we're all standing outside, saying, "Jack, turn the fucking wheel!"
You know?
(Laughter)
(Applause)
I mean --
(Applause)
It's democracy at stake.
It's our culture at stake. It's our world at stake.
And Twitter is amazing and shapes so much.
It's not as big as some of the other platforms,
but the people of influence use it to set the agenda,
and it's just hard to imagine a more important role in the world than to ...
I mean, you're doing a brilliant job of listening, Jack, and hearing people,
but to actually dial up the urgency and move on this stuff --
will you do that?
JD: Yes, and we have been moving substantially.
I mean, there's been a few dynamics in Twitter's history.
One, when I came back to the company,
we were in a pretty dire state in terms of our future,
and not just from how people were using the platform,
but from a corporate narrative as well.
So we had to fix a bunch of the foundation,
turn the company around,
go through two crazy layoffs,
because we just got too big for what we were doing,
and we focused all of our energy
on this concept of serving the public conversation.
And that took some work.
And as we dived into that,
we realized some of the issues with the fundamentals.
We could do a bunch of superficial things to address what you're talking about,
but we need the changes to last,
and that means going really, really deep
and paying attention to what we started 13 years ago
and really questioning
how the system works and how the framework works
and what is needed for the world today,
given how quickly everything is moving and how people are using it.
So we are working as quickly as we can, but quickness will not get the job done.
It's focus, it's prioritization,
it's understanding the fundamentals of the network
and building a framework that scales
and that is resilient to change,
and being open about where we are and being transparent about where are
so that we can continue to earn trust.
So I'm proud of all the frameworks that we've put in place.
I'm proud of our direction.
We obviously can move faster,
but that required just stopping a bunch of stupid stuff we were doing in the past.
CA: All right.
Well, I suspect there are many people here who, if given the chance,
would love to help you on this change-making agenda you're on,
and I don't know if Whitney --
Jack, thank you for coming here and speaking so openly.
It took courage.
I really appreciate what you said, and good luck with your mission.
JD: Thank you so much. Thanks for having me.
(Applause)
Thank you.