We've had one or two comments and people asking the questions about how
how easy or how difficult it must be to deanonymize data, so it's a problem, isn't it?
Absolutely, I mean I think a few years ago 2013 there was a published study
where they took 1.5 million phone records for in a small European country,
probably Belgium from the one of the authors.
And they randomly selected four locations
they would sort of locate the nearest cell tower and from that they could uniquely identify, re-identify
95% of the database and by choosing 11 data points they got 100%
They were able to uniquely identify who these people were.
Underlying that is, you know, some simple observations about people tend to live somewhere, work somewhere
and even those two data points are probably enough to uniquely identify folks.
And then you can look for everything else tagged with that and you can probably track people around.
So it's actually lots of examples of where what seems like a large amount of random data is not
because it actually contains within it this huge amount of structure.
It's always difficult to imagine how one can anonymize the data set
and it's create a data set that has records which are associated with an individual
where we essentially just delete the name or replace it with some code, a random number.
It's very difficult to defend that against all possible future attacks
because someone could turn up with another data set which includes, I don't know, three or four of the fields
that are the same as the anonymous data set but with a name associated with it.
And then by correlating those two data sets together you'll get a set, may not be a one-off,
but you might get a set of candidates of
"I Think the following four people, the following four anonymous records could be this person because they match on some other fields"
And then you could start saying "Well, let's look at then an anonymous record what the other fields are
and maybe there's something entertaining or scurrilous about that or privacy invasive that people wouldn't want to know".
And this is very much the attack we saw on the film rating database where
people rated films with their names on them and then an anonymized data set was published
with so and so rented all these you know this anonymous person rented all these films
give us a better recommendation algorithm for them. So there's a big record of all these anonymous identifiers.
But of course you watch a film and you write a review,
so even though this set of people who had published their reviews had not reviewed every film they watched,
the fact that you could correlate they obviously rented these movies and then
made their reviews probably shortly afterwards causes you to be able to take that and public identifier
and associate it with all the movies they've watched including the ones perhaps they didn't want people to know about.
The classic is politicians getting, you know, caught watching adult content and that's very embarrassing for them.
And the thing is that if someone says here's anonymous data where the records still contain all the data
And it's associated with an individual or maybe we could call that pseudo-analyzed
It's very difficult to protect against all possible future databases that might arise
with real names associated with them. And that's the very that's the problem with the
deanonymization is a threat that's always there.
So, that's a fundamental problem if what you have is a record per person,
no matter what you do. I think there's another there's another thing as well which is perhaps
mentioned in that space: sometimes the way in which people choose the anonymous identifier is pretty pretty dire really.
So New York cab company decided to publish all the pick up an drop off points of all the yellow cabs
and they anonymized the badge number of the driver.
Now in fact there are just aren't that many badges, so someone computed all the possible hash values of the badges and hence
you're able to just invert the whole thing and so you're able to reidentify. Because there was an algorithmic means
to produce the "anonymous identifier". In fact what you really want to do is just
If you do want to do this you really just need to then generate a random number.
Do not make it a hash value when the space of values is not that large.
Brute force attacks. It's a bit like cryptography
you know
It was really hard, it took a lot of work to break enigma during the Second World War, right?
But these days your average PC would crack it in seconds if that.
So, It's slowly increasing, you know, processing power that's available, and the fact that we have
GPU cards with hundreds of GPUs on them or even more so cryptographic accelerators as we see in some of the
some of the modern processors. If you have all of that those engines there
then the challenge is that you simply have to keep making the problem harder and harder.
So the next thing you could try to do is say well
Why don't we do the sort of techniques that the Census adopt which is they don't publish per postcode data
because even that's a too fine grid they they aggregate the data out such that the statistics that they publish
are at the levels of you know hundreds of thousands of people and that doesn't completely prevent
re-identification, but it certainly starts to make it substantially harder.
And some of the times like cryptography the mission is to make it substantially harder and to put it beyond
Reasonable computational resources. After all all cryptography falls into that category.
It's not like you cannot break cryptography, you just do brute force.
It just may take a thousand years, but remember: what was predicted to take a thousand years
in 1945 is now, you know, milliseconds on a modern computer so you always go to factor in that.
So the issue is how difficult to make this for people to re-identify.
And of course essentially, you know, when you get down to it
This is what intelligence agents would be doing for years, I mean, they'll be looking at all sorts of
random sources of information and understanding how they all correlate together.
So, even when you are aggregated, it may still be possible to to re-identify within that group.
And it has to be not just that you aggregate to within say a hundred people or something,
it also has to be that there can be no population of I don't know less than ten or twenty in that group.
I mean my son was asked to fill in a survey at school, and they said "Oh, it's all completely anonymous"
and they asked for his postcode. Now he sat at dinner long enough listening to me go on about this repeatedly
And he simply he then put in a bogus postcode
And he said "Well, that's ridiculous, because there's only two of us kids at school of a school age on this street".
And I said "No, it's worse than that, son. He lives around the corner that's a different postcode
because of the different street name". There's precisely one Fifteen year old child on this street.
So it was uniquely identifying. And something that someone thought "Well that will be fine",
you know, it wasn't because actually although there are a number of people living on our street,
there's only one there was only one fifteen year old at the time, so it was uniquely identifying.
So, It's a complex problem even when you aggregate to try to anonymize because you've got to think about
is it a unique sample within that aggregate in which case it's still a problem
So you find that the people have been doing this for years, which is people like
the Office of National Statistics, they have tried and tested mechanisms
where they've evaluated the risk of re-identification and that's how they publish things like census data
and the data that they publish every month that comes out of the ONS. They're very careful about them.
Now, this is something that actually has become, recently I've noticed become very
interesting to lots of companies who are trying to deal with data protection:
the new General Data Protection Regulation coming in the EU.
Where they're saying "Well what on Earth could we possibly do
because we were allowed to keep these synonymous records", but what does that mean?
And I heard a very sensible colleague from a large company say:
"Well, why don't we do what the ONS does because they've had a
hundred years experience on this and they seem to know what they're doing and
surely we can carry the same level of risk as they can". So it is a very subtle problem
but like most of these things in the same way that I would simply say that
there is no cryptography that is completely bulletproof, it's a question of just how difficult is it to crack the crypto
When it comes to anonymization the question is how difficult is it to break the anonymization?
And he's simply applying new interesting and different perspectives and different data assets
to the original one, you know, things that the original creator never thought about
will be the attacks so our mission always has to be simply to make it at least provably
challenging computationally to do
And that's and it should personally maybe some genius is gonna quote someone that really does work,
but I've yet to see anyone who would believe that we truly can anonymize in the same way that
other than one-time pads in cryptography we can't do a crypto that's not open to brute force attack.
Doesn't really matter, but they started discriminating against that and a BitTorrent was the classic where lots of people were discriminate against BitTorrent
But then of course it turns out that many people were using BitTorrent for perfectly legitimate reasons.
Yes, there were BitTorrent file sharing and there still is plenty of it around