MARTIN OMANDER: Hey, Wes.
WESLEY CHUN: Hi, Martin.
What are we doing today?
MARTIN OMANDER: Well, last week, you showed me your weather
alerting app.
You said you'd like to optimize it
by pulling the values from a cache
instead of hitting an external API every time.
WESLEY CHUN: Yeah.
And I read up on the new CPU allocation
controls in Cloud Run.
So let's do that.
[MUSIC PLAYING]
MARTIN OMANDER: All right, Wes.
Could you walk us through your weather alerting app?
WESLEY CHUN: Yes, of course.
When I enter a US state abbreviation in this text
field, the app hits api.weather.gov to get
the weather alerts for that state
and then displays them to the user.
MARTIN OMANDER: Nice.
WESLEY CHUN: As you can see, it's a web app.
And I didn't want to maintain any servers
or virtual machines.
So I deployed it to Cloud Run.
It's running pretty well.
MARTIN OMANDER: So Wes, I'd imagine weather alerts don't
change second by second.
So is there an opportunity to cache these alerts?
WESLEY CHUN: Yeah.
That's a great idea, Martin.
By caching those weather alerts, the app
can respond faster to users and our bandwidth bill is reduced.
We also use up less computing resources for our friends
at weather.gov.
MARTIN OMANDER: Sounds like a win-win-win.
WESLEY CHUN: Yeah, that's right.
So I want to cache the weather alerts for all 50 states.
Well, I could schedule a job with Cloud Scheduler
that updates this cache every five minutes.
But now with Cloud Run, and it's always on CPU,
we don't have to do that.
Normally, Cloud Run throttles the CPU
after your service has responded.
But with this new feature, we can
keep the CPU at full capacity for background tasks,
like updating my cache.
And on top of that, we get a discounted rate, too.
MARTIN OMANDER: I can see how an always-on CPU would
be useful for other things beyond updating caches, too.
For example, OpenTelemetry tries to send
tracing data to the server when the CPU is less busy.
WESLEY CHUN: Yeah, that's right.
This is useful for [? goroutines, ?]
node async calls, Java threads, Kotlin coroutines, and Spring
Boot background functionality.
MARTIN OMANDER: I'm sold.
Let's do it.
WESLEY CHUN: All right.
Well, first, I'll edit my Cloud Run
service in the Cloud Console.
Then I select "CPU is always allocated" and then redeploy.
MARTIN OMANDER: Wow.
That was easy, Wes.
WESLEY CHUN: Yeah.
And now in my code, I'm going to add a loop here
that updates the cache every five minutes.
MARTIN OMANDER: And you've implemented updateCache
further down in the code?
WESLEY CHUN: Yep, down here.
Take a look.
So I added all of this after the standard get and post
processing.
Any viewers who are interested can see the implementations
in the linked repo.
MARTIN OMANDER: Looks good.
Time to deploy?
WESLEY CHUN: Yep.
And here goes.
MARTIN OMANDER: All right.
Looks like it's done.
I'll use the service to get the weather list for California
now.
It looks like we didn't break anything.
What do the logs say?
WESLEY CHUN: Let's see.
Well, here, I see that the cache is
being updated for all 50 states outside of any requests.
And here, I see that the service returned a response to you
and that it was read from the cache.
It's working.
MARTIN OMANDER: Very nice.
How much will we pay for this?
WESLEY CHUN: Well, regarding costs,
note that we are being billed for more time, meaning rather
than just the time when instances respond to requests,
we're now incurring costs for an entire instance's lifetime,
meaning from when a container is started
to when it's terminated.
On the other hand, we get a big discount--
25% less for CPU, 20% less for memory--
and we won't be charged per-request fees
for either requests or background tasks.
MARTIN OMANDER: I like that.
How does CPU allocation interact with Cloud autoscaling?
WESLEY CHUN: Well, Cloud Run's autoscaling is still in effect
and may terminate container instances if they aren't needed
to handle incoming traffic.
Also, an instance will get shut down
15 minutes after processing its last request unless it's kept
active using main instances.
Once your instance gets shut down,
any background work will also get killed.
In general, always-on CPU is good for high and even loads
or if you want to run background tasks.
Those background tests could be your own code or libraries
like OpenTelemetry or Spring Boot.
MARTIN OMANDER: Good to know.
Thanks, Wes.
And thank you for watching all the way to the end.
If you have any questions for Wes
or me about CPU allocation in Cloud Run
or feedback on this episode or suggestions
for future serverless topics, please let us
know in the comments.
Bye for now.
[MUSIC PLAYING]