>> On the special build edition of the AI Show,
learn more about the differential privacy research
that powers White Noise from Salil Vadhan,
leader of the Privacy Tools Project
at Harvard University. Make sure you tune in.
[MUSIC].
>> Hello. I'm Salil Vadhan,
a computer science faculty member at Harvard University.
I'm excited to tell you about differential privacy, White Noise,
the system we are building in collaboration with Microsoft,
and OpenDP, the larger
open source community effort it is feeding into.
Before jumping into the science,
I'd like to mention that my own start
in differential privacy came with
two wonderful visits to the Microsoft Research Silicon Valley lab.
With just two months in 2008,
my collaborations with Microsoft researchers,
Cynthia Dwork and others led,
to my first four papers on differential privacy,
a topic which I am still working on to this day.
What is the motivation for differential privacy?
It was the realization that
the publication of what seems to be even innocuous,
aggregate statistics can lead to severe privacy risks.
What do we mean by aggregate statistics?
These can be publications of
statistical tables as done by government agencies,
like the Census Bureau,
the releasing of trained machine learning models
as done by tech companies,
and it can also include interactive query systems,
where analysts can present queries of their own,
like a regression they'd like to run and get back results.
Such query systems are often used in practice,
and hope that not giving analyst
direct access to sensitive data will protect privacy.
However, we now know that we have to be very careful,
even with statistical aggregates.
In the early 2000s,
Dinur and Nissim proved a mathematical theorem saying that
the publication of too many accurate aggregate statistics,
each one just a simple random count,
allows an adversary to reconstruct
almost the entire underlying sensitive dataset.
The US Census Bureau internally applied a form of this attack
to the billions of statistics they released from the 2010 census,
and discovered that they could reconstruct and
re-identify a significant fraction
of responses from individual households,
motivating them to modernize
their disclosure limitation methods and adopt
differential privacy for the 2020 census.
Related vulnerabilities have been found for
genetic data and machine learning models.
To hammer this point home with
a concrete example, a couple of years ago,
Reddit users discover that if you entered
nonsense into Google translate,
it would start spitting out passages from the Bible verbatim.
Why did this happen? It turns out that the millions
of parameters in the deep neural net doing translation,
which again seem like statistics
aggregated over huge training dataset,
had inadvertently memorized some of the training examples.
Now this may not be so worrisome
when it's passages from the Bible,
but imagine a system trained on e-mails,
or personal images, or medical data.
What's the message from all this?
It's not that all statistical releases are always dangerous,
but rather we need a quantitative to theory
to tell us how much is too much.
How to ensure that we are not publishing
too much statistical information too accurately,
and be sure that it can't be combined to yield such attacks?
Differential privacy offer such a quantitative theory.
In doing so, it aims to allow statistical analysis,
such as drawing inferences about
a population from which a dataset is drawn,
training machine learning models that generalize,
while mathematically ensuring that
individual level data is not leaked,
no matter what attack strategy is used by an adversary and
what kind of auxiliary information an adversary has in hand.
How is differential privacy achieved?
It is achieved by injecting small amounts of random noise into
statistical computations to obscure the effect of each
individual while still allowing
useful statistical signal to come through.
In a bit more detail,
differential privacy requires that even if
an adversary is interacting with the system,
choosing queries to try to learn
about a specific individual in the dataset,
what it sees should be essentially the same
as if we had removed that individual's data
from the dataset and replaced it with arbitrary other values.
More precisely, for any two dataset,
these datasets that differ on one individual's data,
like a dataset with your data in it
and one with your data removed or modified,
the probability distribution of results seen
by the adversary should be essentially the same.
How close these probability distributions are is
given by what's called the privacy loss
parameter, often denoted epsilon.
The smaller epsilon is,
the greater the level of privacy protection and
hence the greater amount of noise that needs to be introduced.
This is the sense in which
differential privacy offers a quantitative theory of privacy.
I don't have time to tell you about how
differentially private algorithms are constructed,
which can be quite sophisticated.
But already as a six years ago,
there was a wide theoretical literature showing that many data
analysis tasks can be done in a differentially private manner,
and this has only exploded since then.
There are also some remarkably general theorems,
saying a very wide range of statistical estimation and machine
learning problems can be done with
almost no asymptotic cost for differential privacy.
That is, the amount of error due to
the noise introduced for privacy vanishes
more quickly than the sampling error
as the size of your dataset grows.
Another amazing theoretical possibility is that of generating
rich synthetic data with differential privacy.
That is, publishing a dataset consisting of
artificially constructed records that preserve
an exponentially large collection of
statistical properties of the original dataset,
yet does not leak
noticeable information about any individual record
from the sensitive data.
There remain significant challenges to make
these remarkable theoretical results
on this in the previous slide practical.
But they tell us that there is a lot
we can hope for from differential privacy.
Beyond all of this beautiful theory,
we have also seen a number of
exciting practical deployments of
differential privacy in recent years.
This includes the landmark decision by
the US Census Bureau, that I mentioned earlier,
to use differential privacy for the 2020 census,
as well as a number of deployments by large tech companies.
All of these, however, are organizations with lots of
in-house technical expertise to develop
their own customized differential privacy solutions.
So a goal of the work we are doing and our collaboration with
Microsoft is to enable wider adoption of differential privacy.
There are a number of challenges to achieving this,
and our efforts address a number of them.
We've been working towards this at Harvard in
our interdisciplinary Privacy Tools Project since 2012,
and we're excited to now be working with Microsoft
on the White Noise system that is designed to
be an end-to-end system that can allow
non-experts to make effective use of differential privacy.
We're also building on this experience to launch
a larger community open source software project
in differential privacy called OpenDP,
in which collaborations between
academia and industry can bring ideas from
the research literature into
production quality software that can be used by all.
One use case, which has been a driver for our efforts at Harvard,
is the incorporation of
differential privacy into data repositories,
such as the Dataverse software that
others on our Harvard team have built and is used by
social scientists and researchers around
the world to share their datasets with others to verify,
replicate, and extend their findings.
Unfortunately, today,
most data repositories are not
equipped to take on privacy sensitive data,
and when they do, it is not made available for download.
The only way to get access to
such data is through a manual application and approval process,
which can take months of negotiation between institutions.
Our goal is to enable wider sharing of
sensitive data while providing strong protections of privacy.
We plan for a future version of White Noise to have
a simple graphical user interface
that a researcher coming to deposit data
in a Dataverse repository can use to release
differentially private statistics about their dataset
without having any prior expertise in differential privacy.
Then another researcher can come, explore these releases,
including the uncertainty coming
from the noise introduced for privacy,
as well as make queries of their own.
To summarize, statistical releases do carry real privacy risks,
and differential privacy provides
the only principled way we know for avoiding them.
There is a rapidly advancing science of
how to get increased utility with differentially private releases,
and we are all working together to turn
this science into software. Thank you.
[MUSIC].