MARTIN OMANDER: Hey, Craig!
I had a great time playing board games last night.
Which reminds me, I've recently completed a top-secret side
project, a web app to track each player's performance
and win rates over time for bragging rights.
CRAIG LABENZ: Ooh, bragging rights.
I love bragging.
But say, what did you use to build this web app?
MARTIN OMANDER: So lately I've been using Django
for my web apps' back ends.
But deploying Django isn't always easy.
CRAIG LABENZ: Martin, when I say--
I feel you on that.
But luckily I've been working on a Terraform script
to automate the entire process on GCP.
Want to test it out on your project?
MARTIN OMANDER: Yeah, that would be amazing!
Show me the way.
[MUSIC PLAYING]
CRAIG LABENZ: I've been using and deploying
Django for over a decade.
And honestly, it's tricky every time.
MARTIN OMANDER: That makes me feel better,
to hear you say that.
There are a lot of tedious parts to get right.
CRAIG LABENZ: Yeah, there sure are.
You need a database, a web server that ideally scales,
storage for static files and media,
and encrypted application secrets.
It's not easy to get all that right.
Now, Cloud Run is serverless, so it can handle the scaling part.
But the rest are still a challenge.
MARTIN OMANDER: And your script handles
all of that, including application secrets?
How is that even possible?
CRAIG LABENZ: It's possible by the magic of,
we'll probably have to make a few edits
to your settings.py file.
But I've been using it on my own web apps for a bit now.
Here, let me run it real quick to redeploy my application,
and then we can integrate it into your project.
MARTIN OMANDER: That sounds great.
Oh, look, there it goes.
Wow.
You and I just stood here like bumps on a log
while the script did everything.
CRAIG LABENZ: I know.
Wasn't it great?
I'll send you the link to the repository.
And after you clone it, you can walk through it.
MARTIN OMANDER: OK, I have the code up.
I've seen a little Terraform before, but I'm not an expert.
What am I looking at here?
CRAIG LABENZ: Terraform is an open-source
infrastructure-as-code tool that allows
for consistent, declarative workflows
to manage large-scale application infrastructure.
That's a bit of a mouthful.
But in short, it just means you get
to write down what servers your application needs
and how they should connect to each other.
And Terraform figures out how to make that a reality.
MARTIN OMANDER: And it knows how to talk with Google Cloud?
CRAIG LABENZ: It sure does.
Looking up at the top of the file,
the first block specifies a dependency on the Google Cloud
platform bindings.
So with that, Terraform downloads the necessary code
to translate everything that we write here
into Google Cloud API calls.
MARTIN OMANDER: Wow.
OK, I'm hooked.
Can you walk me through this file?
CRAIG LABENZ: Yeah.
So the next step-- make environment variables
locally accessible to your Terraform workflow.
You can hard-code anything that's
fixed to be default values and then leave
the rest to be passed in from the command line.
MARTIN OMANDER: OK, that makes sense.
And what does the script do next?
CRAIG LABENZ: If you've ever configured a new GCP project
by hand in the Web Console, you've
encountered the process of activating specific service
APIs.
So this next part looks ahead of the complete set
of service APIs it will require and turns them all on.
We'll also want to set up a custom service account
to run our Cloud Run service.
The default service account has access to do a lot of things.
So I usually create my own service account
and specifically set what it can and cannot do.
MARTIN OMANDER: I love that this script even
captures the principle of least privilege.
CRAIG LABENZ: Always.
Now, the next step takes some time
to complete the first time you run this configuration,
because it initializes an entire Cloud SQL Postgres instance.
MARTIN OMANDER: This section looks
like it does a lot of heavy lifting
I see the creation of a Cloud SQL instance--
let's see, a user, and a specific password.
But how do we get that password into our code?
CRAIG LABENZ: Hold on to that thought,
because it is coming soon.
But first, there's a section that
sets up the final project-specific values we
have, our admin user's password and a Google Cloud Storage
bucket to host our static and media files.
MARTIN OMANDER: Oh, neat.
I like that Terraform generates these secrets,
so they never exist in plaintext.
CRAIG LABENZ: Yeah, exactly.
And now that Terraform has generated everything sensitive,
we can prepare to securely get all of that information
into our Django settings.
MARTIN OMANDER: OK, that one is a little dense.
Can you walk me through it, Craig?
CRAIG LABENZ: Absolutely.
This block uses a simple template, which I'll send you,
to create an environment variable
file with our database credentials, secret key,
and storage bucket.
Later, we'll make an edit to your settings.py file
to actually read in those values.
MARTIN OMANDER: Perfect, because I
have flagged some to-dos in my settings
file for each of those things.
What's next here?
CRAIG LABENZ: So far, we've generated values
for a few sensitive pieces of our app.
But Terraform has yet to actually share
that information with anyone.
So this part creates a new entry in Secret Manager
and gives access to the necessary service accounts.
MARTIN OMANDER: I see this creating new secrets.
But have our secret values been uploaded to Secret Manager yet?
CRAIG LABENZ: Yes, good eye.
If our Terraform manifest ended here,
we would, in fact, have some empty and pretty useless
entries in Secret Manager.
But that's where this next part comes in,
to gather up all of the values we care about
and populate those secrets.
MARTIN OMANDER: Ah.
And I'm guessing that our Django app will then
need to access these secrets during startup?
CRAIG LABENZ: You're really quite a good guesser, Martin.
Has anyone ever told you that?
MARTIN OMANDER: Aw, shucks.
CRAIG LABENZ: So this is where that template file comes in,
because the structure of the secrets that we uploaded
isn't just the values awkwardly placed in the text file,
but instead are valid environment variables file
that our application can source, thus allowing your Django
code to pull those values right out of the environment.
MARTIN OMANDER: Wow.
Terraform's thought of everything.
And now we're getting to the Cloud Run service definition.
CRAIG LABENZ: Indeed.
It's taken a lot to get here, but the Terraform manifest
is now finally ready to define our actual Cloud Run service.
MARTIN OMANDER: I also see another IAM setting there?
CRAIG LABENZ: Yeah.
There's always more IAM setting.
In the Google Cloud Console, there's a checkbox that says,
"Make this service public."
And the GCloud CLI, it's the --allow-unauthenticated flag.
In short, these two just open up our service
to the entire internet, which is, of course, what we want.
MARTIN OMANDER: And speaking of policy, it
looks like you're also setting service permissions.
And is that a for loop?
CRAIG LABENZ: Yeah, it is.
Terraform allows you to perform actions
against a list of values.
In this case, we're giving both Cloud Build
and our actual Django service account access to Cloud Run
and to Cloud SQL.
MARTIN OMANDER: But why does Cloud Build need access
to the database?
CRAIG LABENZ: Well, that's a good question.
As most Django developers know, some of your deployments
involve schema migrations that must be applied
running manage.py migrate.
The only problem is, Cloud Run has nowhere
to run that one-off command.
So to solve this, I've set up a series of steps
in the cloudbuild-migrate.yaml script to run against
our container image.
One of these is the famous migrate command.
And thus Cloud Build, will also require access to the database.
MARTIN OMANDER: So is that your continuous delivery
configuration?
CRAIG LABENZ: Yeah, it is.
And you can run this manually or as part of a deployment
to automatically run the steps that you need
to migrate your application, or any other commands
that you need.
MARTIN OMANDER: All right.
There's one last part of the Terraform
script that we haven't covered, these output parts here.
What are they?
CRAIG LABENZ: Those are values that Terraform
will display once it's finished applying everything.
And these are the useful values that we need,
our service URL and our superuser password.
Of course, we've defined our password as being sensitive,
so it will only output that if we explicitly ask for it.
We could get those values by running GCloud commands
against Cloud Run and Secret Manager,
but this saves some time.
MARTIN OMANDER: Wow.
This is all so declarative.
CRAIG LABENZ: Yeah, that's the point.
Terraform allows you to capture your entire deployment
process in one declarative manifest,
and it handles the rest.
MARTIN OMANDER: Now, all that mumbo-jumbo about consistent,
declarative infrastructure really makes sense.
So with all this, I can go edit my settings.py file now.
CRAIG LABENZ: Sure can.
Open up that file, and we'll get started editing.
MARTIN OMANDER: OK, I'm in my settings.py file,
and I'm ready to grab my passwords from Secret Manager.
CRAIG LABENZ: Great.
I'm sending you a snippet now, which
you can drop in at the top of your file, pretty much as is.
MARTIN OMANDER: Oh, this is hefty.
Let's see.
That read_env function will read from a local environment,
plus a .Env file, if I have one.
But then we get to this try block.
Let's see.
We attempt to use the Google Auth API to load our default
credentials in production.
I know these are provided by Cloud Run itself.
If they're missing, we must be doing local development,
so it quietly fails?
CRAIG LABENZ: Yep, you're on the right track.
This setup handles both production
and local development, which is nice.
MARTIN OMANDER: And then, if it sees we have a project ID,
we use the Secret Manager API to get those very same secrets
we saved in Terraform manifest.
And then, with those values, it loads them into the environment
as well?
CRAIG LABENZ: Yeah, precisely.
And instead of accessing those variables via os.environ,
like you might be used to, I do recommend
using the environment library to streamline the process.
MARTIN OMANDER: This sounds amazing.
Let me quickly edit my settings.py file
in the other locations to read in all of our secret values.
First, I'll update my secret key.
Then I'll let the environment package handle
all of my database settings.
And lastly, I'll read in the correct static files bucket.
OK, so Craig, you're telling me that if I run this script now,
it will deploy my app?
This task that I usually spend an entire day suffering through
will be just done?
CRAIG LABENZ: Almost.
Open up that readme, and you'll see a few extra commands
that we have to run the first time we deploy our project.
MARTIN OMANDER: OK, opening it now.
All right.
I'm setting the $PROJECT_ID variable
and activating it with the GCloud CLI.
I'm already authenticated with my project.
Otherwise, I would need to run GCloud Auth application default
login.
Enabling a few basic services, I see.
I imagine this simplified the Terraform manifest in some way.
Next up is gcloud builds submit.
I know this one.
It actually reads my Docker file and uploads the resulting image
to Google Container Registry.
Oh, and here we go, terraforming it and terraform apply.
Whoa!
Look at it go.
Things are moving pretty fast, except--
oh, yeah.
You said the database would take a while to initialize?
CRAIG LABENZ: Yeah.
While that's working, want to see something neat?
Open up this project in the Google Cloud Console,
and you'll see this database initializing exactly
as if you'd just gone through the Create Cloud SQL instance
wizard.
MARTIN OMANDER: Oh, right!
Look at that database being created.
Like you said, this can take a while.
See you back here in five minutes?
CRAIG LABENZ: Sounds good.
MARTIN OMANDER: All right.
The Terraform script finished successfully.
And look, just like you said, it's
printed my Cloud Run Services URL.
But first I have one more first deployment command to run.
CRAIG LABENZ: Yep.
But you'll actually run this command any time
you deploy a change that alters your static files
or your database schema.
MARTIN OMANDER: And now, the last step--
we output the URL and check that it worked.
Oh, my gosh!
I can hardly believe it!
CRAIG LABENZ: Pretty neat, eh?
And now let me show you something else cool.
Run terraform apply again.
MARTIN OMANDER: But Craig, I don't need two instances
of my app deployed.
CRAIG LABENZ: No, of course not.
But trust me.
Run the command again and see what happens.
MARTIN OMANDER: Well, you've never led me wrong before.
So here we go.
CRAIG LABENZ: See?
It detected that everything you declared already exists,
so the whole process did nothing.
And now, moving forward, you can run
that command whatever you want.
In each pass, it will only do something if you just
changed the Terraform manifest.
MARTIN OMANDER: Craig, I honestly can't believe it.
With this Terraform script, I can deploy my entire Django app
to Cloud Run in no time.
This usually eats up my entire Saturday.
CRAIG LABENZ: Happy to help, Martin.
But say, maybe next time as a thank-you,
when we're playing Catan, you cannot play such hardball with
me when I propose resource trades.
MARTIN OMANDER: You can't just expect
me to offer you buy one, get one on my lumber
whenever you need it.
[MUSIC PLAYING]