[MUSIC PLAYING]
OK, so if you've been following along series,
you've heard me talk about Cloud DLP and all the ways
that it can help you inspect your data
and de-identify it with a whole bunch of different techniques.
We talked about the concepts, and we saw some code.
But if you're wondering what DLP looks
like from a pointy, clicky perspective,
stick around for this extended episode,
where I'll show you some DLP functionality in the Google
Cloud Console.
[DOOR CLOSING AND LOCKING]
[MUSICAL CHORD]
So for this demonstration, we're going to do a few things.
What we have is a GCS bucket with few files.
I'm going to show you how to create an inspect template
and then kick off an inspection job that's
going to tell us which files might
have some sensitive information in them.
Then after that, I'll show you how
to create a de-identification template
and then use that to create an identical data set
in BigQuery with encryption applied
to some sensitive fields.
OK, so before we start, let me just
show you real quick what we're working with.
So I have a GCS bucket, some data.
And in that bucket, we just got some text files, CSV files.
Some of them have sensitive values.
Some of them don't.
Like, for example, we have sample_s10
has some phone numbers, while s20 has names
and Social Security numbers.
So it's a mixed bag.
We've got full names.
We have social security numbers.
We've got phone numbers.
So what we're going to start with
is a DLP inspection template.
Let's get rid of this guy.
So creating a template here is pretty simple.
You go to the Data Loss Prevention
page, Create Template.
And here, you'll choose either an inspect template
or a de-identify template.
We'll go for inspect for now.
OK, let's call our bucket_inspect template,
a bucket.
All right, hit Continue.
And now, here's the part where we
choose what infotypes we actually want to inspect for.
So I'm just going to go for--
since we saw that.
We saw Social Security numbers, and let's add a phone number.
Boom, added, done.
And then here, we choose the confidence threshold
or likelihood.
So it can go, again, it can go from very unlikely to very
likely.
We'll just hit Possible, right in the middle.
Hit Create.
Give that a second.
All right, so here is our inspection template.
And here are the two infotypes that we're looking for,
the likelihood right there.
So now that we have that template,
we can go over here, back to the Data Loss Prevention page,
and we can now create a job or a job trigger.
All right.
So the cool thing about DLP is that you
can make it inspect on a schedule,
so we'll call this our weekly_inspection.
And what are we going to inspect?
It's going to be Google Cloud Storage.
And you can see that you can also
do BigQuery or Cloud Datastore.
All right, and we'll make our bucket name sample_data_dlp.
All right, and here, if you scroll down a little bit,
you can actually choose to use sampling.
So we're not going to use sampling for this example.
But if you have a very large data set,
you might want to use sampling, because the more files it scans
for, the longer that job is going to take
and the more resources it's going to take up.
And then when you do choose to use sampling, you can always
adjust it here, whether you want to sample 50% or 20% or 100%--
which wouldn't make sense-- but 100%.
But for our purposes, we're going to say no sampling.
Here, you can also choose the files
that you want to limit your inspection to.
We'll just leave it at All for now, and hit Continue.
And now, we choose our template.
We only have one template, so it's
going to autopopulate our other stuff, the templates that we
have.
And there it is, our bucket_inspect.
And here are the details.
Again, here are two infotypes and the likelihood of possible.
So for each job, aside from just a template,
you can also add some infotypes here.
But again, the more infotypes you look for,
the longer that job is going to take.
So what I'm going to do is just deselect
any additional infotypes, so we have nothing.
We'll leave the likelihood, again, as possible.
Hit Continue.
And now, here are the actions that you want your job
to take after it's done.
So we're going click here, save to BigQuery,
and I'll show you why that's important in second, but--
All right, our project, a-dlp-demo.
Our data set is bucket_inspect.
So I created these data sets ahead of time in BigQuery.
So before you do this, you'd want
to do that yourself, create your own data
set and then also an empty table.
Inspect.
And there are other actions that you
can add to each job, like Publish to Pub/Sub or Notify
by Email.
But for now, we'll just go to a BigQuery.
Hit Continue.
And here, you can choose when you want it to run.
So you can run daily, weekly, or custom, so we'll keep it
at weekly.
Hit Continue.
And that's going to give us a recap of our job configuration
in the JSON objects.
Scroll down past that.
We're going to hit Create, and we're going to Confirm.
So now, it's creating that trigger.
Since it's a weekly one, it's not running yet.
So we just hit here, Run Now.
Boom, and they're running trigger weekly inspection.
So an inspection job could take some time,
and that depends basically on how big your data set is
or how many infotypes you specified in that template.
All right, so we must see how that job went.
You want to go back to the Data Loss Prevention page,
and under Inspect Jobs, there it is.
Click that, and this is pretty much a recap page
of how the job went.
So you'll see that it found 52 findings.
52 findings were found in your data.
And 44 of them were phone numbers, and 8 of them
were Social Security numbers.
So there's not much more information
on this page than that.
If you want more information about each of the findings
that it made, you want to see what
it threw out in to BigQuery.
So these are all the fields associated with each finding
that it made in your data, so I'll show you a quick query.
We're going to select the file name, which
should be here, relative_path.
And we want to include--
what else-- info_type and likelihood.
Hit Run, and there you go.
So it's not just the file has sensitive values in it,
and it made an entry in a database.
It actually made an entry for each finding that it made.
So, for example, you'll see that sample_s08 is here
a few times because it has a few phone numbers.
And you have the option of including
the actual value of that data, but I didn't
enable that for this example.
But you can see how much sensitive data a file
might have rather than just whether or not
it does have that sensitive information in it.
An inspection job is a great way to find out if your data might
have any sensitive value in it.
Now, let's create a de-identification template
and see what we can do with that.
OK, so before we create a de-identification template,
I just want to show you something real quick.
Since we're going to use tokenization
to de-identify information, on their cryptographic keys,
I'll show you how to create a key ring
that I named dlp_keyring and a key that I named dlp_key.
And we're going to use it in a second.
So let's jump back to Data Loss Prevention.
Once again, we're going to go up here, Create Template.
This time instead of Inspect, De-identify.
We'll call this bucket_deid, and this is our Bucket
deidentification template.
Great.
Hit Continue.
And here, we'll pick a transformation rule.
And there are some familiar faces in here.
We see redaction, replacement, replace of infotype.
Down here are our three tokenization methods.
We're going to choose Cryptographic Deterministic
Token.
Down here, we're going to paste in that key information,
the resource name as a path.
And here, it's going to want a wrapped key,
so it's pretty much going to want
a key that's been encrypted by that encryption key.
And I'll show you how to generate that real quick down
here.
I'll open up our Cloud Shell terminal.
And we're going to set this variable to openssl rand
-base64, 16 characters, boom.
And then here's what that looks like, right?
So we're going to take that.
We'll take off any spaces.
And we're going to pipe that into gcloud kms encrypt.
And then we're going to give it our key ring
and our key information.
So location was global.
Key ring was dlp_keyring.
Our key, dlp_key.
And we're going to specify plaintext-file,
which is actually going to be the value that we
piped into this command.
And then we're going to specify a ciphertext-file, which
is the output that we want it to go to, so ciphertext.
Hit Enter.
And what we got now is a file that has some cipher text.
So we can't paste that in here, so let's encode it to base 64--
hold on, let's clear that up--
base64, no wrapping, and ciphertext.
All right, so let's take this guy,
carefully select it, put it on a clipboard, paste it over here,
and boom, there it is.
All right, so let's get rid of this
and continue on with the template.
So with Surrogate InfoType, again, this the value that's
going to prepend in front of the encrypted values.
So since we're only going to look for Social Security
numbers, I'll just make it SSN, and we'
going to specify the infotype down here.
Manage InfoTypes, OK, there it is, US Social Security number.
And again, you can choose up to 150.
But for now, Social Security number is fine.
And we'll hit Create.
So the cool thing about the deID templates,
you can go over here to test it out to make sure everything's
cool, everything's working.
So a Social Security number is three digits, two digits,
and four digits.
And that is an example of what it's
going to look like when it's transformed,
and there's our surrogate value right here.
So to demonstrate de-identification
in action, what I'm going to do is create a dataflow job,
which is going to take our CSV files from our bucket
and then make a de-identified version
of that same data in BigQuery.
So to create a dataflow job, let's head over
here to the Navigation menu, scroll down, Dataflow, Create
Job from Template.
So a job name, let's just call it Bucket de-identification,
deID for short.
And under Dataflow Template, we're
going to choose Data Masking/Tokenization from Cloud
Storage to BigQuery Using Cloud DLP.
And now, we must specify that bucket
that I showed you earlier.
I have a sample_data_dlp, and we want
to target the CSV files that are in there.
So I don't know if you noticed before,
but there was a data set in BigQuery
that was called bucket_deid, totally empty, no
tables or anything.
This will create the tables for us.
And our project ID, dlp_demo.
And the template name, which is on my clipboard,
temporary location is going to be in the same bucket slash
temp.
Great.
All right, let's look this over.
So now what's happening is that dataflow
is going through all of those CSV files in our bucket
and then creating an identical data set
in BigQuery with those values de-identified
using tokenization.
So once the dataflow is done doing its thing,
here is what it's going to look like in BigQuery.
So here's our data set, bucket_deid.
And now, in that data set, there's
a table for each CSV file that was in that bucket
that I showed you earlier.
So if you remember before, we looked
at sample_s20 that had a bunch of names and Social Security
numbers.
Let's query that table and see what we get.
And boom, there it is.
Just like before, we've got name, Social Security number,
and some other data.
But now, those Social Security numbers
are completely unrecognizable, because they've
been encrypted with the key that we
specified in the de-identification template.
So now, this data is essentially safe
to use for any analytic workflows that might
exist in your organization.
And that's it.
That was just a quick taste of what
cloud DLP is capable of doing.
But if you have your own stories of how cloud DLP has helped
you secure your data, feel free to share that in the comments
below.
Thanks for watching.
Until next time.
[MUSIC PLAYING]