- I don't need to start this presentation
by telling you how much of a challenge
this pandemic has been.
We have all been living it every day.
The scientific community has risen to the challenge
at a speed that is breathtaking.
This chart shows the growing size
of the literature.
We're now at well over 25,000 papers,
letters and preprints.
The number's growing daily.
But this chart tells us a lot of stories,
of fast turnaround times,
of doctors, of researchers who have spent their lives
studying similar viruses,
and have now turned their attention to this one.
And also of labs, of scientist,
who have completely retold and refocused
to rise to this challenge.
This chart tells us about a mass mobilization,
but there's also a catch.
A growing information glut
that is hard to make sense of.
Life and death decisions weigh on the question,
which papers do you pay attention to?
Which do you trust?
I work for an AI company called Primer.
We use natural language processing
and machine learning,
to structure large unstructured datasets.
And to summarize them,
to help make sense of them for people.
When the outbreak started,
we refocused our efforts.
And we built this public, free website
in order to give people another way to explore
this growing literature.
We shared it with scientist, with researches,
with policy makers,
and I hope that after this talk,
you'll have an opportunity to go and explore it too.
I hope that you'll find it useful.
Now I'm gonna spend a very short amount of time
talking about how we built it,
and then a longer period
talking about the lessons that we've taken from building it.
We started by collecting the literature.
Firstly, we went to PubMed,
which is a large collection of published,
peer reviewed, papers and letters.
Now, these are from all kinds of different journals.
We also went to the three preprint servers.
bioRxiv, medRxiv, and arXiv.
And we supplemented these with a stream
of news and Twitter data,
linking them up when they mention the papers by URL.
We set up a process to get this every day
and we feed it through a pipeline
of natural language processing.
Doing things like extracting people's names,
extracting quotations and attributing them,
resolving them, summarizing.
And that's what gave us this site.
Now, there's a number of different ways
to explore the site,
and I'm not gonna talk about all of them.
But, I am gonna show you a couple
of my favorites.
So firstly, we have a dashboard
that gives you a view of the most recent changes,
so what's new.
We then have a fully automated natural language briefing.
So, this is generated without human intervention
every single day.
And it gives you highlights of recent papers
that are widely shared and discussed.
And my favorite feature on this,
is that we're able to find quotations
by paper offers,
when they appeared in the news talking about those papers,
and show them alongside.
We use classification to bucket the papers
into major research categories,
and we use an unsupervised topic modeling
to discover emerging areas of new research.
So, what did we learn from this?
Well, this is going to be a tale of three parts.
Of confusion, of conspiracy,
and finally, thankfully, of hope.
Science is not a tidy process.
It leaps forwards,
and fits and bounces,
and especially in a pandemic
when the available data grows rapidly.
These ideas are challenged.
Our best interpretation keeps on changing.
This is a well known researcher
walking back his earlier findings,
on how the virus made its way to Seattle.
After taking more samples,
after doing more research,
what they realized was there another path
that perhaps made more sense,
through British Columbia.
This is how science is meant to work.
But it does make it very hard to keep track of.
To know the current status of each idea,
and to weigh the uncertainty of what we know now.
And that's especially difficult for policy makers
who have to make decisions now.
But this is happening at a time
when the scientific process is changing.
Preprints are new to the biomedical literature.
So, these are documents
that are not peer reviewed before publication,
and instead that review from other scientist
happens in the public eye
after they are shared.
Now, this means that a research can be shared
much more quickly.
And, a much wider set of eyes can look at them.
And this is really, really good news
in a pandemic where you want...
Where time is of the essence.
But it does make it very hard to know what to trust.
So, we've had preprints around for a long time
in the mathematics and physics world.
arXiv has been around for decades.
But medRXiv, so in the medical space in particular,
it's very, very new.
medRxiv was only large last summer,
we're still figuring out the API's.
We haven't had time yet to figure out
how to talk about these publicly.
But they are a large proportion of the research.
So, 25% of the content on our site
is from these preprints servers,
and a third of the news talks about them.
So, they are very much in the public conversation.
But this is also happening
at a time when journalism is under distress.
As a result of the pandemic,
and of longer term changes and challenges,
40% of journalist in the US were laid off this year.
40%, that's a staggering number
and it means that there are far less people in newsrooms,
able to take the time to help us make sense
of what is going on.
And what does that mean?
Well, here's a paper that was published in early May.
In it, the researchers looked at samples of the virus
that they took from different patients,
and they compared them.
They compared the genetic sequences.
And what they found was
that there were differences,
there were mutations between them,
and that some of those mutations,
some of the versions were much more prevalent than others.
So, what do we take away from that?
Well, the first interpretation.
The first interpretation is that this means
that there is a newer, scarier version of the virus,
that is wildly transmissible.
Or, maybe it means that things are different,
we should be concerned.
On the other hand,
there is much nuance reporting,
that explains that possibly what we're seeing
is just the result of chance.
Some versions just happen
to be the ones that spread further.
So, our interpretation affects how nuanced
we think the science is,
and how much more there is to learn.
But there are also less benign processes at play here.
One of our findings has been
that the most widely shared papers on Twitter,
are the most controversial,
not the most useful.
This paper, which was posted at the end of January,
was at the top of that leaderboard
until about the middle of May.
In it, the researchers compared
the genetics sequence of the Coronavirus,
with a genetic sequence of HIV.
And what they found was,
that snippets of the genome,
the Coronavirus genome,
look the same as snippets of HIV.
This they called uncanny,
and begged the question how did they get in there?
Well, as soon as this paper was released,
the scientific response was immediate and very harsh.
Here's one example,
an assistant professor
of biochemistry from Stanford University,
she said, "The similarity is spurious,
"not higher than chance."
As a result of this,
and of the overwhelming feedback,
the authors withdrew it.
This was celebrated as a great moment for science.
This is how it's meant to happen.
Anyone can upload their research to the preprint servers.
The scientific review happens,
and if the science is found to be faulty,
the paper can be withdrawn,
and then we can all move on.
Apart from if we don't all move on,
we found that of the 42,000 tweets about this paper,
20% of them happen after the paper was withdrawn.
And even now,
if you search for this paper,
what you find are tweets like this one,
asking whether the withdraw
is actually evidence of suppression.
These tweets feud into a wider conspiracy theory.
How do we have this conversations about uncertainty,
moral view updates,
when there are people willing to jump on it
and interpret it this way?
So where do we find hope?
Well, it helps to go back to this incredible chart.
Science is marching on.
We know so much more about the virus now
than we did when this started.
We know how it affects our bodies,
we know how it's transmitted,
and we have many more ideas
about how to treat it.
Science is a fantastic process of finding knowledge,
even though it is confusing.
It's a fantastic process because it works.
But, that doesn't help us hence the question,
of who do we trust?
How do we make sense of this
as we see it unfolding before our eyes?
Well, the first good thing to look for
are the people helping explain it to us.
So, great journalism is able to engage
with the nuance and the uncertainty.
This article from Ed Yong at The Atlantic,
does that very well.
There's no clear evidence,
and their probably won't be for months.
This is the article about that confusing mutation study
that I showed you earlier.
He's written a fantastic series of articles
that explains some of the uncertainty
that we have to live with,
as we walk through this pandemic.
There were also a large number of scientist
who are taking a lot of time,
to explain and contextualize the research that we're seeing.
In the site,
if you go and look for the people who are most quoted,
you click on the numbers,
you can read some of their recent quotes.
One of my favorites here is Caitlin Rivers.
She's able to bring her background in public health
to explain what the path out of lockdown might look like.
Here she is describing how Liberia
handled the Ebola epidemic.
And she's hopeful about what we can achieve now.
So here it is,
we should take time to explore and engage
with this literature.
And we should do so as optimist,
skeptical ones.
Skeptical optimist, engaging with this growing content,
and realizing that it's uncertain,
but looking for the places
where emerging scientific consensus is growing,
where papers build upon each other,
where the evidence is mounting.
So what license can we take away from this more broadly,
and especially for our work as data teams.
Well firstly, it's very good for us to reflect
on the fact that this is the era that we live in.
One in which information is abundant
and easy to access.
But one, where we don't know why we're shared,
the content that we're shared.
Why do we see what we see?
We don't know how it fits into a bigger picture.
You and I, could spend all day reading true things,
and at the end of the day,
not have read any of the same things
in common with each other.
And have a completely different view
of what's important.
That gulf between us is an enormous problem.
It's one of the defining challenges of this era.
But, I do see some reasons for hope.
We have a tool kit in front of us,
where we can do things like structure text,
aggregate it, compare it, summarize it.
We can do that at scale.
Recent advances and natural language processing
allows to do it with much more precision
than never before,
recent advances in the infrastructure
allow us to do it with a much greater ease
than we've been able to do before.
And those may seem like some of the building blocks
that we need,
to be able to build tools,
that really support people's quest for knowledge
and for truth.
And especially for people
whose job it is to make sense of all of this for us.
Whose time is tight and incredibly valuable.
I think we have a real responsibility in this moment
to see what we can do to bring to bear these technologies,
to make better sense of it.
Thank you.