Welcome to 'Back to Basics'.
I'm Krupanidhi Jay, a Senior Solutions Architect with AWS.
In this episode, we are going to talk about chaos engineering with Fault Injection Simulator,
a fully managed service for running fault injection experiments on AWS,
which makes it easier for you to improve your application's performance,
observability, and resilience.
With Fault Injection Simulator or FIS, you set up and run experiments
that help you create the real world conditions needed
to uncover application issues that can be difficult to find otherwise.
AWS FIS provides templates that generate disruptions
and the controls and guardrails that you need
to run experiments in your production environment.
Before we dive in, let's look at some of the challenges with distributed systems.
Distributed systems have multiple components located on different systems
and vary greatly in implementation.
As distributed systems grow in complexity,
so does the difficulty in predicting failures.
Failures of mission critical distributor applications
can cost costly outages.
Even simple distributed systems are complex.
Let's take a look at the simple client server communication.
There are a lot of steps involved to complete a single round trip successfully.
Client puts a message into the network,
the network delivers the message to the server,
the server validates the message and updates its state if required.
Server puts the reply onto the network.
Network delivers the reply to the client,
and client validates the reply and updates its state if required.
There are so many permutations of failure that can happen in the simple distributed system.
Now, imagine all these over thousands and millions of requests.
Traditional testing like unit and functional tests are required,
but don't always address the complexity of a production environment.
Running these tests in isolation only verify a known condition.
What about the random errors we are not expecting?
The configuration drifts, the network error,
the unknown conditions.
How can we test for something which we don't know about?
Luckily, that is a practice which helps with these kinds of unknowns,
and it is chaos engineering.
Chaos engineering is the process of stressing an application
by creating disruptive events such as server outages, APA throttling, etc.;
observing how the system responds and implementing improvements.
We literally break things on purpose to build a more resilient system.
Chaos engineering is not about injecting faults randomly without a purpose,
but injecting faults in a control environment
with well-defined experiments
to build confidence in your application and tools to withstand failures.
To do that, you have to follow a well-defined scientific method
that will take you from understanding the steady state of the system
you're dealing with to articulating a hypothesis, running an experiment,
often using fault injection, verifying the results,
and finally learning from your experiments in order to improve the system.
While chaos engineering is a great option to have in your toolbox,
it's often hard to get started with it,
hard because most of the time you have to stitch different tools,
scripts, and libraries together to cover the full spectrum of faults
you can inject in the system.
Compatibility of required libraries and agents
for these open source tools may be limited.
It is also difficult to ensure a safe environment in which to inject results.
You want to be able to limit the extent of potential issues
from an experiment and without guardrails, an experiment can go sideways
affecting the rest of your environment.
And finally, some failures are really hard to reproduce.
It is important to realize that outages rarely happen because of a single failure.
It usually happens because of a combination of small failures
all happening at the same time or in a sequence.
To address these concerns, we launched AWS Fault Injection Simulator.
You can use AWS Fault Injection Simulator
via the AWS Console or the AWS CLI.
The Console is great for getting started and trying things out,
while the AWS CLI lets you automate things with your CI/CD pipeline.
AWS FIS integrates with IAM rules and policies,
so you can be sure that only authorized parties can run experiments.
You can also define more fine grain controls,
for example, what type of faults are authorized and on which resources.
When you create your chaos experiment from the console,
AWS FIS automatically creates a template for you,
a template you can export and reuse later.
We start with an experiment template, which will comprise of different port injection actions
like generating a high CPU load on an EC2 instance.
FIS actions also help you control the timeline and duration of the experiment.
FIS performs the actions on the AWS resources that are specified as targets
when you start the experiment.
A target can be a specific resource, like an EC2 instance or RDS database
in your AWS environment, or one or more resources that match a criteria you specify.
You can monitor the experiment using AWS CloudWatch,
and FIS can also be integrated with Amazon EventBridge,
which allows you to integrate with your existing monitoring tools.
Experiments once started automatically stop when all the actions are complete,
or you can optionally configure to stop the experiment
when an alarm or event is triggered.
Once the experiment is complete,
you can view the results of the experiment to identify any performance,
observability, or resilience issues.
You can use AWS FIS in periodic gameday events
by creating potential failure conditions
and observing how effectively the team and systems respond.
You can also integrate FIS into your continuous delivery pipeline.
This will enable you to repeatedly test the impact of fault actions
as part of your software delivery process.
In this episode, we explore how you can run chaos engineering experiments
easily using AWS Fault Injection Simulator
FIS provides you the flexibility and built in safeguard mechanisms
to run chaos experiments at scale.
Check out the links below for more details.
See you next time.