[MUSIC PLAYING]
SPEAKER: Welcome back.
In the last episode, Kevin from GreenMobility
told us about immutable deployments.
So do immutable deployments apply
to other components of your system
as well besides Cloud Run, Kevin?
KEVIN: Yeah, we want to run jobs everywhere
in our production environment.
And for example, you can get into trouble
with your cron jobs.
Developers might add or modify your jobs
in the cloud schedule, and after a few weeks,
you're not sure what jobs are there and what they're for.
SPEAKER: Ah.
How do you prevent that drift for Cloud Scheduler jobs?
KEVIN: Let me show you.
[MUSIC PLAYING]
We're using Terraform to deploy Cloud Scheduler jobs.
And that way, all job definitions
are in a Terraform manifest file that's in source control.
It's easy to see when a job was added and why.
Whenever we apply a Terraform manifest,
Terraform compares the list of scheduled jobs in the manifest
with the jobs in production.
It then adds all the list of jobs in production as needed
to make sure that production contains only the jobs listed
in the manifest.
SPEAKER: Nice.
That ensures that the production environment stays consistent.
What else are you doing to keep production under control?
KEVIN: We are also focusing on security.
We needed to provide transparency and be
strict about who can see what.
For example, access groups-- we have a BI and data.
They can run reports.
We have a DevOps group.
They can deploy code and view production.
And then, we have a dev group.
That can't view production, but they can check in code.
SPEAKER: Ah, but Kevin, if your developers don't have access
to the production environment, how
do they know what settings and code would work there?
KEVIN: We have many dev projects.
Developers are free to experiment with settings
and test their code in those projects.
And every now and then, we destroy the dev
projects and create new ones using the settings
in our production project.
SPEAKER: Ah.
As you went full serverless, is there
anything that worked differently than you expected?
KEVIN: Yeah, the Cloud Scheduler.
For example, it wouldn't start new jobs
if old ones were running.
So our solution to get around that
was just using two Cloud Schedulers.
It's actually cool, because it means that the scheduler keeps
track of what is running, which can
prevent some really bad bugs.
For example, if two instances of the same job
are trying to modify the database at the same time,
for example, you could get into really serious trouble.
SPEAKER: Mm, that makes sense.
KEVIN: Also, we find that many of our jobs run at five minutes
past the hour, every hour.
That puts a lot load on our cloud SQL database.
We could manually schedule the job start times over the hour,
but we have a lot of scheduled jobs, and we keep adding more.
It would be nice if Cloud Scheduler could just
run our jobs randomly for us.
SPEAKER: I will let the project manager
for Cloud Scheduler know.
Is there anything else you've done
to be more productive with Google Cloud?
KEVIN: Some tasks I perform often,
like switching between projects and users,
requires a lot of clicking and waiting in the Cloud Console.
I wrote two command-line tools, dpc-run and dpc-auth-user,
to quickly navigate between projects and users.
They were easy to write, as you can
do so much with the [INAUDIBLE] commands from a shell script.
Feel free to use it.
SPEAKER: Nice, I will check them out,
and I will put the link in the description below.
Any other learnings, Kevin?
KEVIN: We also need to sync tens of thousands
of records between our system and an external system,
and this happens a few times per hour.
At first, we had a system dedicated
to running this, which was slow and complicated.
So what we did was we split this big job
into 500 small jobs using Cloud Tasks, which
meant we could run them asynchronously in parallel.
We found the Cloud Tasks made it easy to kick off these 500
jobs quickly, and also, the code that
ran each of these 500 small jobs became simpler.
SPEAKER: 500 jobs?
Does it that make it harder to handle errors?
KEVIN: It actually makes it easier.
Each job is shorter, so fewer things can go wrong.
And if any of the 500 jobs fails,
Cloud Tasks tries it again.
We don't have to write any of the retry logic ourselves.
SPEAKER: Oh, nice.
So Kevin, what's the verdict?
Does serverless get a thumbs up or a thumbs down from you?
KEVIN: A big thumbs up.
I could have built the data just in backends, scheduled jobs
and so on using virtual machines,
but our team has saved so much time by doing it serverlessly
with Firestore Cloud Run, BigQuery Cloud Task, and also
Cloud Scheduler.
It means that we have been able to add
more functionality to our cloud sharing platform quicker.
SPEAKER: Good to hear, Kevin.
Thanks for sharing your serverless
experiences, and thank you, everyone, for watching.
If you have questions for Kevin or me,
please add them in the comments.
Also, this episode was a little different from the others
in the Serverless Expedition Series.
Please let us know if you found it useful,
and be sure to like this video and subscribe to the channel.
Until next time!
[MUSIC PLAYING]