>> Tune in to this week's episode of the DevOps Lab.
We have our returning guests, Ned Bellavance,
to walk us through how we're going to
validate our terraform code and use
GitHub actions to deploy our resources into Azure. Tune in.
[MUSIC]
Welcome to this week's episode of the DevOps Lab.
With us this week we have a guest that's been here before.
Welcome, Ned.
>> Hey, April. Thanks for having me back.
I guess I didn't do too bad the first time.
>> I love having you on the show, Ned.
Always great to have you back as a fellow Hashi Ambassador,
a fellow East Coaster.
What's not to love about you?
>> What's not to love about being a HashiCorp Ambassador?
It's fantastic program and nominations just close.
We're going to find out soon if we made the cut for next year.
>> I know. I'm like,
please let me make it.
I do all great things, but we'll see.
Good luck to you-all that.
Welcome back. For people that haven't
seen your awesome face here before,
can you tell them what is you do?
>> Sure, my name is Ned Bellavance.
I'm the founder of Ned in the Cloud LLC.
I'm a HashiCorp Ambassador, like you said,
and what I do is create educational technical content.
That's my main focus in life.
>> You're here today to show some cool stuff with
Terraform GitHub actions and some code validation. Am I right?
>> Absolutely correct.
Last time when I came on,
we put together a GitHub Actions workflow to
deploy terraform code against Azure.
We were standing on some Azure infrastructure. That was the idea.
We want to do it in a way that wasn't too scary.
We didn't try to add too many bells and whistles on it.
If I could take you through the current workflow as we set up in
the previous demonstration and then we'll see
what improvements that I made in the whole process.
Moving over to my slide deck
and don't worry, there's not too many slides.
In our current workflow we had
a main branch and we want to make a change to our infrastructure.
We're following a GitOps workflow.
We'd create a feature branch and then within that feature branch,
we'd start making commits.
That's where GitHub actions would kick in.
Every time we made a commit and pushed that commit up to GitHub,
it would kick off a GitHub action,
but it wasn't doing anything yet.
It was just there. It was like,
yeah, I can do something.
As we made additional commits in our feature branch,
every time GitHub actions like, I'm ready,
I'm going to do something, but we're like,
not yet, hold your horses.
Once we get to the point that we're ready to deploy this update,
we'd create a pull request to
merge this feature branch back into the main branch.
That would kick off a GitHub action and in that one,
we would run terraform plan to plan out the changes.
Then we would add those proposed changes as
a comment in the pull request that was open.
We could see these changes look
good and then if we like the changes,
we could merge the pull requests
and that would kick off another GitHub action,
which would in turn, run terraform
apply and apply those changes to our infrastructure.
That was the basic workflow we set up not to scary,
we took it step by step.
Now, we can take it another step further and start
validating our infrastructure as code.
The way we're going to do that is to start adding
some actions to our commits.
In our updated workflow,
we're going to take advantage of two things
that terraform has built-in,
terraform format checking and syntax checking with
the commands terraform fmt and terraform validate.
Fmt just make sure that you're
following HashiCorp's standard formatting guidelines.
If you're not, it will actually automatically format for you.
We don't want to do that. We just want to check.
We're going to check the formatting
and then we're going to run terraform validate,
which actually checks the syntax of your code to make sure it's
valid against terraform and its providers.
We're going to run that every time we commit
an update and push that update to the feature branch.
Then when we get to the pull request,
now we're going to introduce another tool.
We're still going to run format and
validate to make sure that's still good.
We'll run plan to get the plan into
our comments and then we're going to run a tool called checkov.
April, have you ever used checkov? Are you familiar with it?
>> I have. I've used a quite a few times Ned.
>> Excellent. This isn't news to you,
but it might be news to the audience
>> I'll pretend. I'll act surprised.
>> Cool. What is checkov?
It's a solution created by bridge crew that
does static code analysis of infrastructures code.
That sounds like a lot of words.
Let me try to condense that down a little bit.
It takes a look at your terraform code and then it has
a whole bunch of policies and best practices,
and it compares what's in your code to what's in the policies
and best practices and flags your code and says,
this doesn't line up,
this shouldn't be here, or you should add a setting.
It makes recommendations.
That's essentially what it does.
You can use the policies they have out of the box or you
can write your own custom policies
that make sense for your organization.
That's essentially what checkov does.
Now that we know that, let's walk through the updates
to our workflow in GitHub actions in that YAML file.
The first thing that I
mentioned is we're going to run terraform format.
This is the step in our GitHub actions
workflow that'll run the terraform format.
We only want to run this on feature branches.
The if statement says GitHub.ref
if it's not equal to main, run this.
The thing I want you to run is terraform fmt -check and check
just returns zero if there's
no formatting problems and one
if there's any formatting problems found,
so it would fail if they found formatting issues.
Then the next thing was terraform validate.
For that one, again,
we only want to run this on feature branches.
We'll say GitHub.ref is not
equal to main and we're adding another thing in here, which is,
and success or failure,
which looks at the previous step,
which was our terraform format and says,
I don't care if that was successful or if it failed,
I still want to run validate on top of that.
We get to run both tests every
time we push a commit up to the future branch.
Then the last one is Checkov.
This one's a little more complicate,
but the if statement says we want
to run this when there's a pull request.
We're going to use a GitHub action
called Checkov from bridgecrewio,
and then the with statement gives it
all the arguments that you would use if
you were using Checkov at the command line,
and we're skipping a bunch of checks here.
There's a lot of checks in Checkov,
as the name implies,
and we don't want all of those
applied because it would be ridiculous.
I filtered some of it out to make it a little
easier for us in the demonstration.
That's all the changes to our GitHub actions,
so do you want to take this for a spin?
Should we jump over to them?
>> Before we go, I think we should talk about
why this is important, right?
>> Oh, yeah.
>> Why do I want to do all the things?
Why do I want to validate my code?
Because this is going to pull up all my errors that I've
made as a human, true.
But that's exactly why we want it,
because so many times we make changes to production,
we don't want to do that in production,
or if we're going to a development environment,
or free product,
or UAT environment, we want to know how we bring in the code,
and this helps us write
better code because a lot of people that are
using Terraform may not have a true developer background,
they might not be an engineer, they might work in
infrastructure apps and aren't comfortable with these things.
These help remediate the human errors,
and we want to run tests on our code
because early and often that's what we do in DevOps.
This is really important to do for
everyone and when we build this into the pipeline,
it means we don't have to do it,
but it helps us fix what we've messed up and write better code.
>> Yes, and I make mistakes all the time.
I'm always running Terraform format,
and validate before I try to do anything,
and this just does it for you, which is nice.
We know we're pushing quality code.
Let's walk it through it's paces and see it in action.
I'm going to switch over to
visual studio code where I have this already loaded.
Here is visual studio code,
and we have the main portion of our code in the file main.tf,
because I'm real creative that way.
You'll notice there's some formatting issues
in this file already,
on line 35,
the spacing is not quite right,
and there's a validation issue on line 30.
I'm referencing an attribute
of the Azure Resource Group that doesn't exist.
Hopefully both of these things are going to fail.
The branch that I'm on is ADO-LABS,
so once I commit and push this up to GitHub,
it should kick off the GitHub action against this branch.
I'll just call this adolabs test,
and go ahead and sync that up,
and then we will switch over to the browser and I'll show you
what's happening with GitHub actions in the browser.
>> Cool.
>> We're over in our GitHub repository,
and if you want to try this out for yourself,
you can go to this public repository and fork it,
and it has directions for you to set this all up yourself.
If we go over to actions,
there's the adolabs test,
and sure enough, it failed.
If we want to know why it failed,
we can look a little bit deeper.
We can see it failed on two things,
it failed on Terraform format and Validate,
and just looking at the format one,
it says error,
there's an unsupported attribute,
you're referencing something that doesn't exist,
you probably need to fix that.
Let's go back to visual studio code and fix our two errors.
The one thing we need to fix is the spacing in here,
I'll go ahead and fix that and I'll remove
the bad reference and add the good one and save it,
and I think that should do it.
Now we made those changes,
call it fix, and hopefully that fixed all of our changes.
We can go back to the browser and now with any luck,
it will have passed successfully.
If I go into "Actions" now it's running.
Here we go. It's going to run through Terraform format
and Terraform Validate and hopefully come
back clean on both of those.
>> This is going to run against your code before
it gets officially pushed and checked into the repo,
so it's that protection against repo,
you can trace where those errors are,
and you can back out and fix them before they
become a bigger problem in your code base.
>> Exactly. Now Terraform Format has a check,
Terraform Validate has a check, and we're green.
I fixed my errors.
Now I'm ready to open
a pull request to merge this branch into main.
I'll go ahead and click on that and create the pull request,
and once I create a pull request,
that's going to kick off the next set of GitHub actions,
so we can see now there's a new actions running adolabs,
and this one is going to run through the Checkov process.
There we go, it's pulling the Bridgecrew,
and it's going to run through
all the steps and then eventually get down to Bridgecrew.
Lots of stuff scrolling by. Now, like I said,
Bridgecrew has a ridiculous number of checks and policies.
The first time you run it,
if you don't tell it to skip anything,
it's going to come back with 30 or 40 errors on your code,
and that might be a little scary at first.
You can filter some of those out,
they may not be applicable, and that's what I did.
It's running through Terraform plan now,
and there we go, it did fail in Checkov.
That's that's difficult to look at,
so let's go back to the pull request and
remember this is being added as a comment,
so if we go down and look at
the comment that GitHub action's added,
it says everything was successful except for the Checkov,
and if we look at the Checkov results,
it tells us here's the two errors that you have.
It's saying you have to ensure that the web app redirects
all HTTP traffic to HTTPS and ensure secrets are encrypted.
We can easily go back and fix one
of those if we go back to our code.
I have it commented out right now,
but I could uncomment HTTPS only and save it,
and then commit it and push it,
and now that one would be valid.
However, as I said,
Checkov is always adding new things to the policy.
Looking in here, there's one forget
to make sure that all secrets are encrypted.
I haven't actually seen that one before,
so this would fail on another push,
so now I have some research to do to go figure out,
is that something I need to actually work on?
Or is that something that I can now ignore?
It's probably something I should pay attention to,
because encrypting secrets is important.
That is our updated workflow to validate our infrastructure as
code using the built in
Terraform tools and using the tool Checkov.
>> Awesome, Ned. Thank you.
Well, so while Ned goes and creates a GitHub issue and
resolves this Checkov errors, you-all can tune in,
go to the repo that Ned has published,
it is public for you to take on and replicate what he's done in
users workflows and get started using
GitHub actions to validate your Terraform code.
Put all those in the show notes.
Ned, thank you again for coming on today.
It was always a pleasure to have you on the DevOps lab.
Thank -you everyone for tuning in.
Tune in next time to the DevOps Lab.
[MUSIC]