JENNY BROWN: When it comes to gaming, whether it's
action, RPG or simulation, staying in the action
means being online.
But in the offline world, you rep your fandom with merch.
Will Critter Junction's e-commerce shop
be able to handle their hype?
Stay tuned to find out.
[MUSIC PLAYING]
Without any in-person conventions
this year, Critter Junction players from around the world
have begun to flood their site to purchase character
cards, apparel, and other swag.
Critter Junction has been testing App Engine standard
to run their new merchandise shop,
Because App Engine auto scaled their application
across multiple instances to meet the demands
of additional traffic.
But during load testing, they used Cloud Trace
to measure response latency.
And they noticed a higher than usual latency
when they used heavier concurrent requests
to their service's HTTP end point.
You see, pending latency is how long
a request can be sitting in the queue
before App Engine decides to spin up another instance.
If all of your app instances are busy when a request arrives,
the request will wait in a queue to be handled
by the next available instance.
As the load increases, this means requests
are processed more slowly.
App Engine will then start a new instance
to help based on limits you set, like CPU utilization,
throughput, and max concurrent requests
of the current running instances.
But what they didn't know is that App Engine also
needs to load their app's code into a fresh instance when they
redeploy a new version of their app,
and when maintenance and repairs of underlying infrastructure
or physical hardware occur.
Though cold starts on App Engine are rare,
the first request or loading request sent to a new instance
can take longer to be processed, because the instance first
has to load your app's code, including
any libraries and resources needed to handle the request.
This means a gradual increase in response times
to handle new traffic.
What you want instead is to get the initialization
to happen sooner, before a new instance serves live traffic.
You can do this by using a warmup request, which
loads application code into an instance ahead of time,
before any live requests reach it.
App Engine attempts to detect when
your app needs a new instance and initiates a warmup
request to initialize it.
Any new instances accept requests
after they finish loading your app's code.
New requests can then be handled faster.
Warmup requests are used by the App Engine scheduler, which
controls auto-scaling of instances
based on your configuration.
App Engine issues get requests to /_ah/warmup.
And then you can implement handlers for this request
in your code to perform application-specific tasks,
like pre-caching app data.
For most supported languages, just add the warmup element
under the inbound services directive
in your app.yaml file.
Then create a handler that will process requests
that are sent to ah.warmup.
Your handler should perform any warmup logic
that is needed by your app.
Let's walk through a Go example.
Our function performs the required setup steps
for the application to function, and it
logs when an App Engine warmup request is used
to create the new instance.
These warmup steps happen in setup for consistency
with cold start instances.
Everything else remains the same.
This setup function executes the per instance one-time warmup
and initialization actions.
Finally, the index handler responds
to requests with our greeting.
One thing to note is warmup requests
don't work in all cases, despite the best-effort attempt
to send requests to already warmed-up instances
when enabled.
So you might still face loading requests,
like if the instance is the first one being started up
or if there's a steep ramp-up in traffic.
In these cases, you should use the resident instances, which
you can learn about below.
Once warmup requests were implemented,
Critter Junction was able to reduce cold start instances
during increases in traffic to their online shop
once convention season was underway.
Stickers for everyone!
Check out the link below for language-specific steps
on warmup requests, and stay tuned for what's
next for Critter Junction.
And remember, always be architecting.
[MUSIC PLAYING]