KRZYSZTOF GRYGIEL: Hi.
I'm Krzysztof from the Google Cloud Autoscaler team.
In this tutorial, I'm going to talk
about how to build scalable applications on Google Compute
Engine using the Compute Engine autoscaling and load balancing
services.
The load balancing service lets you distribute traffic
across many virtual machines, while the autoscaling
service ensures that there are always
enough virtual machines to handle an increase
or decrease in load.
By the end of this video, you will
know how to create a simple scalable
service using both autoscaling and load balancing.
First, we'll set the stage and discuss a scenario
where a user requires to scale the application front end.
Then, we'll go over how Compute Engine can
address some of these needs.
Finally, we will run some actual examples.
Now let's get started.
Imagine you're a developer who has just
built a new web application, and you're about to launch it.
Let's assume that your application consists
of the following components.
A front end web server with an external IP address associated
with your service; a back end server handling
all of the business logic and computation
for the application; and a persistent storage,
such as a MySQL database, where you will store
all permanent application data.
So we have your serving infrastructure running,
and are now receiving requests from your users.
As the user base and traffic to your application grows,
your front end running on a single instance
will soon reach its capacity limit
and won't have enough CPU power to process
more parallel requests coming from your users.
However, since the front end of your application
is a stateless server, it can be replicated
to multiple virtual machines to handle
many simultaneous requests.
Knowing this can help address potential
serving capacity problems.
If you provision more front end instances
and hook it up to your back end system,
you can serve more requests in parallel.
Of course, the back end instance could
be scaled in a similar way, as long as it is also
a stateless server.
That is, if all persistent state lives only in the database.
We now have more capacity, and no request should be dropped.
There is one issue, however.
Right now, you have multiple front end servers,
each with a distinct IP address.
So you have enough capacity to serve the user traffic,
but you need a way to distribute it across these servers.
What you need is a single IP address for all your users
to connect to.
To do this, you can set up a network load balancer, which
exposes a single external IP to the world
and forwards the traffic across multiple front end instances,
ensuring your instances experience uniform load.
As the traffic to your service increases,
you would keep adding front end servers
so that the serving capacity can always
handle the observed traffic.
While this process might work well in the beginning,
you might begin to notice that the volume of incoming traffic
varies depending on time of the day.
You'll also notice that on some days,
you have more traffic than other days.
While you keep adding machines to be
able to serve the maximum number of user connections,
you quickly realize that you also
want to remove servers during periods of low traffic.
So, in fact, your serving needs to look roughly like this.
And you could save money overall if you added and removed
servers in response to changes in traffic,
rather than maintaining the maximum number of servers
all the time.
Here is where an autoscaler can come to the rescue.
Instead of keeping track of traffic
and adjusting the number of servers manually,
you could configure autoscaler to do the job for you
automatically.
An autoscaler constantly keeps track
of how busy a group of instances are by watching the utilization
level of instances.
You define what utilization metric the autoscaler watches,
such as average CPU load, and define
a specific per-instance utilization target,
and the autoscaler automatically works to maintain this target.
For example, you might the set the utilization
target of your instances at 60% CPU utilization.
When the CPU utilization exceeds this target,
the autoscaler adds more instances
to handle the load automatically.
Let's assume you have one instance receiving
one request per second, which consumes 30% of the CPU
on the machine.
If the traffic suddenly grows to three requests per second,
the machine will run hot, using 90% of CPU.
At this point, the autoscaler will add as many instances
as necessary to get it below the target utilization, which
in our example is just one extra instance.
With two instances, the load drops
to 1.5 requests per second per instance,
and the CPU utilization drops to 45% on each instance, which
is below the target 60% utilization that we defined
earlier.
If the utilization remains low for a long time
and it is possible to keep utilization
below the target with a fewer number of instances,
autoscaler will start removing instances.
Now that we have defined our scenario,
let's go over the configuration steps
you would need to set up your resources
like we just described.
First, you need a group of instances.
To create it, you must define an instance template
which describes the image, machine type, startup script,
and other properties of the virtual machines in the group.
This template will be used to create new instances
in the future, as well, either by the autoscaling service,
or if you manually add a new instance to the group.
Then, create your group of instances.
Use the instance template from step one
to create a managed instance group which
contains any number of identical instances.
Once you have created the managed instance group,
you can enable autoscaling for the group.
Finally, create the network load balancer to handle traffic
from a single external IP.
The network load balancer talks to a group of back ends
called the target pool.
A target pool can contain individual instances or groups
of instances.
In this case, it would just consist of the single managed
instance group you create.
The network load balancer uses a forwarding rule
which defines the protocol, IP address, and port
the users will talk to.
That's it.
Now it's time to try it out in practice.
We are going to use the developer's console
to perform all the steps.
We assume that you already have a Google Cloud Platform
project set up.
For the purpose of this demonstration,
we'll only create the front end layer,
which will respond to HTTP requests
without talking to a back end.
Step one, create an instance template.
Go to Instance Templates pane, and choose New Instance
Template.
At this step, you are selecting machine type, operating system,
and disk for your servers.
Use the default settings.
Give a name to your template, and check Allow HTTP Traffic.
Finally, add a startup script for your VM
in the Instance Metadata field.
For this example, we are adding a startup script
that downloads the front end application
code from cloud storage.
The front end is a super simple web server written in Python
which only simulates the effect of real request processing
by running an idle loop that consumes some CPU
power in response to HTTP get requests.
In a real application, we would replace this idle loop
with some logic to do request processing, communication
with the back end, and to generate HTTP responses.
The code for this demo is available under
storage.googleap is.com/autoscale
r-demo/frontend.py.
Based on the code, notice that our front end will serve
requests under the path /service.
So we will later be appending it to the HTTP
address of the front end.
Of course, if you prefer, instead of using the Python
app, you could install a standard Apache web server
and try the PHP script for a similar result.
Step two, create an instance group
using the template that you just defined, and enable autoscaler.
Select a preferred zone for your service
to operate, and give your group a name.
The autoscaler configuration is pre-filled with defaults.
Here we will use the average CPU utilization of the VMs
as the utilization metric to measure.
The target utilization is set to 60%.
Also keep in mind that the autoscaling policy
can be customized.
Instead of CPU usage, you could choose other utilization
metrics, such as rate of requests going through the load
balancer, or arbitrary cloud monitoring metrics.
For now, we'll just use the defaults.
Click Create to create your instance group.
Creating the instance group may take about two minutes.
Check when the group is ready by watching
the Activities tab in the bottom right corner of the screen.
To see the details of the instance group,
select it in the Instance Groups pane.
Here we can see that the first instance has been created.
Step three, configure load balancer.
Select Network Load Balancing.
You may notice that there is another option called
HTTP Load Balancing next to it.
This is a more advanced form of load balancing, which
offers options specific to HTTP traffic such as balancing
based on URL map.
As we don't need these advanced features,
we are going to use the generic network load
balancing for this example.
Go to the Basic Setup tab of the Network Load Balancing page.
Under Region, choose the same region
where the instance group lives.
The forwarding rule section specifies the endpoint
that your users will send the traffic to.
Leave it set to the default options.
That this, ephemeral IP, port number 80, TCP traffic.
Then we define the target pool.
Recall that the target pool is the group of instances
that the network load balancer will forward traffic to.
We choose Select Existing Instance Group,
and pick the instance group we created in the previous step.
That's it.
You're done setting up your configuration.
You can test your stack by entering the IP address
of the forwarding rule in the web browser,
followed by the /service path.
To get the external IP address of the forwarding rule,
go to the Forwarding Rules tab of the same menu.
It works.
So far, the load balancer didn't have to do much work,
since the front end only had one instance.
For a real test, send some more traffic to the instance.
Let's run a shell script to send HTTP get requests to the IP
address of the forwarding rule.
This command generates one HTTP request every second.
The front end was implemented to consume one second of CPU time
to serve a single request.
So on a single core machine, it will
be running at 100% utilization.
The autoscaler should start another instance soon
to handle the increase in load.
Let's return to the Instance Group page
and select our instance group.
Here we go.
The Instance Group page now shows an additional instance
added by the autoscaler.
Let's try to double their load.
Simply decrease the delay between the requests
to half a second.
We should see the response time immediately going up.
Some requests are even timing out.
In a short while, autoscaler adds another two instances.
The response time drops down to normal.
I encourage you to experiment more with this setup.
You may notice that when the traffic stops,
it takes a while before the autoscaler starts
deleting the instances.
This is to reduce disruption to the service,
and to avoid the cost of frequent restarts
of virtual machines.
We hope that this presentation demonstrated how easily you
can start building your own scalable applications
with Google Cloud Platform.
Autoscaler is designed to work equally well for Google scale
applications, running on hundreds of virtual machines,
to let you take full advantage of the Google Cloud Platform
scale.
[MUSIC PLAYING]