Welcome to 'Back to Basics'.
I'm Adrian De Luca.
Werner Vogels, Amazon's CTO, once famously said,
"Everything fails all of the time."
That might sound like a scary prophecy,
but what he meant was that we need to build systems
that embrace failure as a natural occurrence,
even when we don't know what the nature of them are.
In the old days, developers used to compile their code,
hand over the applications to the operational team
to figure out how to run in production.
Well, it turns out that most of the failures were because we didn't invest
in hardening and automating the best practices.
In the cloud world, if you build it, you run it.
Today, we're going to look at how we do it really well
on Amazon's Elastic Cloud Compute Service, or EC2.
We've got a lot to get through, so let's dive in.
When you launch your application on EC2,
AWS starts collecting lots of metrics in CloudWatch.
Using these services together
makes it really easy to programmatically increase their reliability.
Designing for failure requires you to consider a number of key elements.
First, understanding resiliency of components.
Second, automating recovery from a failure.
Next, you need to also look at scaling to meet demand.
And finally, testing.
Resiliency is basically the ability to recover from a service interruption,
and we apply this to the components that make your application usable to users.
And it includes things like compute, network, and storage.
Availability is measured in percentage using this formula,
and sometimes referred to as the number of nines.
So for example, 99.999% equates to around 5 minutes of downtime per year.
Other measures that we use between dependencies
are mean time to failure and mean time to recover, which can both be estimated.
But more about this later.
In your application running on 1 or more EC2s,
you will have multiple dependencies which affect this.
This could be networks, web servers, databases, or APIs.
I'll cover the most important patterns.
It all starts with some planning.
Deploying your workload to multiple locations is a must.
This is the foundational element of minimizing the fault boundary
for your applications.
So make sure all your critical resources run across multiple availability zones
because this provides physical isolation.
Service quotas ensure that you have enough resources for your application,
as well as don't overstep them.
In any one region, you have 5 Amazon VPCs with 200 subnets.
For Elastic Block Service, or EBS volumes, that are attached to your EC2 instances,
you can set provisioned IOPS to throttle reads and writes of your data.
Amazon EC2 has a limits dashboard that not only allows you to check
on how many resources you're using, but also increase them if necessary.
So let's look at how we recover from some common failures.
Your application on EC2 can become unresponsive for multiple reasons,
perhaps a hardware failure, or a software problem.
So the best way to protect yourself is to have multiple instances.
Using the Elastic Load Balancer or the Application Load Balancer
to route them to the healthy ones is the best way to mitigate this.
Listeners check for connection requests on protocols that you can configure,
like TCP, HTTP, or Websocket ports, and route them to the target groups.
Health checks periodically send requests to test their status.
Configured with timeouts, they manage the availability pool.
This is how we reduce the mean time between failure.
Inability to access data is another frequent reason for failures.
The volumes that users store data on can become corrupted, overwritten,
or simply unavailable through misconfigurations.
So protecting the data volumes is really important.
It starts with identifying two things, Recovery Time Objective,
this is the maximum tolerable downtime for data to be available again,
and Recovery Point Objective, the amount of data that can be lost.
This determines the back-up schedule to implement.
For data that changes frequently and needs to be recovered quickly,
you would want to back that up every hour,
whereas for data that doesn't get accessed a lot,
you can afford to back that up, say, daily or weekly.
Here you can take snapshots of EBS volumes to the Amazon Simple Storage Service, or S3,
and use the Amazon Data Lifecycle Manager to automate it.
Versioning helps you track the copies,
and also it's a good idea to use S3 to replicate it
to another AWS region, just to keep it safe.
Auto Scale Groups monitor your instances
and automatically adjust capacity
to maintain a steady fleet to meet your user demand.
It does this by monitoring CloudWatch metrics,
like CPU and memory utilization.
But for more complex applications, you can create your own metrics too.
I strongly recommend that you use target tracking scaling policies
to scale on metrics like average CPU utilization
or request count per target metric, which the Application Load Balancer uses.
This ensures EC2 Auto Scaling closely follows the demand curve for your application.
You can configure this to optimize for availability,
say, keeping capacity at 70%, or cost, say, at 40%, or a mix of both.
CloudWatch Alarms allow you to define
upper and lower bounds for scaling in and out,
and using step scaling policies to control these size adjustments.
For workloads with consistent traffic patterns,
it's a good idea to enable predictive scaling.
This uses machine learning to evaluate historic demand
and creates a forecast for the next 48 hours to anticipate it.
Designing and deploying resilient architecture is not enough.
You need to test it.
Playbooks are not instruction manuals.
They are proactive planning that you do to document reactive actions effectively.
When failure scenarios not covered by the playbook are encountered
during the testing and production, put out the fire first,
but always make sure that you review processes
and update it with what you've learned.
Deployment needs to be more than just repeatable.
It also needs to be dependable.
This is where CodePipeline can help.
When developers can make changes to a source repository,
CodePipeline automatically detects the changes,
building new artifacts required, and runs tests.
If testing completes successfully,
the built code is deployed to a staging server for further testing.
From the staging server, CodePipeline runs even more tests,
such as integration and load testing.
Simulating user behavior is another way in which we can anticipate failure.
Canary testing allows us to test your workloads from diverse locations.
Here Amazon CloudWatch Synthetics allows you to create canaries
to be able to test your endpoints and APIs.
This is what we call continuous integration or continuous deployment.
So now you've learned some of my top tips for resiliency with EC2.
Check out the links below for more details.
See you next time.