MARTIN OMANDER: Hi, I'm Martin Omander, and today
I'll tell you about a pain point from my previous developer jobs
and how to solve it with modern technology.
I wished I'd had this technology back then.
[MUSIC PLAYING]
Before my current job as a Developer Advocate at Google,
I worked at a string of startups as a software engineer.
I always struggled with how to store
passwords, API keys, certificates,
and other sensitive data.
If I hardcoded these secrets into the source code,
I had two problems.
One, anyone with access to the source repo
could see the secrets.
They could copy the secret somewhere else,
and we didn't have auditing and access controls.
And two, it would be hard to move code between dev, test,
and production environments.
Secrets typically differ between environments.
So in my previous job, I did what the Twelve-Factor App
methodology recommends.
I put the secrets in environment variables.
That way they were not part of the code base.
But the secrets had to be stored somewhere
so I put them in a file, which I had to be careful
not to check into source control.
At one job, we burned secrets to a CD
that we kept in the CIO's desk drawer.
Handling files without source control is awkward.
I remember thinking that there must
be a better way of doing it.
Secret Manager and Google Cloud platform
provides that better way.
It has been generally available since March 2020.
We recently added native integration
with Cloud Run, which makes it easy to mount secrets
in Cloud Run services.
Let's see how we can update an existing Cloud Run
service to use Secret Manager.
The service is a simple API that returns
information about different dog breeds, like this.
The code reads database, user, password, et cetera,
from environment variables when it creates the database
connection pool here.
The environment variables are shown here
in the Cloud Console in plaintext.
So let's say we want a better control and auditing
of the database password by moving it into Secret Manger.
We see here that the Cloud Run servers runs as Dog Breed
API, a custom service account.
I used the principle of least privilege
when I created this account, so it only
has the Cloud SQL client role.
This means that it can connect to Cloud SQL
databases but nothing else.
We will need to remember this account for the next step.
Next, I will go to the Secret Manager in my project,
create a new secret for the database password,
and enter its value here.
Now, let's see who can access this password.
The Dog Breed API is not in the list yet.
Let's add it.
Again, we're applying the principle of least privilege.
The Dog Breed API account currently
has access to db_password, but not any of the other secrets
that we see here.
This reduces risk.
Now, I'm ready to go back to the Cloud Run service.
First, I remove the DB_PASS plaintext environment variable.
I feel better already.
Then, I'll add it back, but instead
of entering the actual password, I
will point DB_PASS to the password in Secret Manager.
Then I click DEPLOY here so this change takes effect.
And look, the API still works.
So, what did we achieve here?
Well first, we can now use the principle of least privilege
for the password and other secrets.
We can define granular permissions
for who can access which secrets and who can add new secrets.
For example, we just configured it
so our Cloud Run service can access, but not write,
db_password in production.
And if only a few people in our dev team
are allowed to handle secrets in production,
we reduce the risk of downtime due to silly mistakes.
I know.
I once mistakenly deleted the production database
at a previous job.
Second, we now have auditing of this password.
By default, whenever it is created or updated
we'll get a log entry like this.
I usually also turn on additional logging
so there will be an entry for whenever
a secret is accessed as well.
And we achieved all this without changing
a single line of source code.
Delightful.
But what if our existing code reads secrets
from a file instead of environment variables?
No problem.
We can configure Cloud Run so that secrets look like files
to the source code.
Pretty neat.
I should also mention there is a third way of accessing secrets
from source code.
Our code could make a call to the Secret Manager client
library every time it needs a secret.
This obviously means that we have to update our source code,
but every time we call the client library
we will get the current value of the secret.
So this method is useful if we expect secrets to change often
and we don't want to redeploy our Cloud Run servers
every time that happens.
The same is true when we read secrets as a file.
If we access secrets as environment variables
and we update a secret, we need to redeploy the Cloud Run
service to get the latest value of that secret.
This has been an introduction to using
Cloud Run with Secret Manager.
The docs have a lot more information,
like best practices for how to manage permissions,
how to rotate your secrets to reduce risk,
and how to version secrets.
Also, in the description below, you
will find a link to the source code I used.
I really hope that this makes it easier
for you to build secure systems.
As developers, our code may be entrusted
with some very sensitive data.
We carry a big responsibility.
Let's live up to it.
Please let me know if this video was helpful
or if you have any questions.
Also, if you have suggestions for new episodes,
please add them in the comments below.
Until next time.