Welcome to another episode of 'Back to Basics'.
I'm Ben,
and today we will talk about
how to control bot traffic in AWS to better secure architecture.
Let's first start by defining what bot traffic is.
Bolt traffic can be defined as any non-human traffic
to your internet-based website or application.
Common bots consist of both good bots and bad bots.
First, we'll go over some examples of good bots.
There are search engine bots such as Googlebots,
that can update their indices of your website's content
to provide end users relevant results.
Also, there are monitoring bots that'll ping your website
to ensure that it's working and online.
However, there are also bad bots such as web scrapers,
vulnerability scanners, and spambots
that you'll want to block before it reaches your infrastructure and application.
Now the best way to protect your application
against this type of traffic in AWS is to use AWS WAF.
AWS WAF is a fully managed web application firewall
that helps protect your web applications or API's.
In general, this is against common web exploits and bots
that may affect availability, compromise security or consume excessive resources.
With AWS WAF, you can block common attack patterns
and customize rules that filter out specific traffic patterns.
AWS WAF also automatically scales based on incoming traffic.
You don't have to pay for when to use,
and the pricing is determined by the number of rules you deploy,
along with the number of web requests your application receives.
To get started, you can create an AWS WAF WebACL
associate it with either your CloudFront distribution,
Application Load Balancer, API Gateway,
your GraphQL APIs in AWS AppSync.
Once you start building out your WebACL,
you can start looking to solutions like the AWS WAF bot Control
or create rate limiting rules.
In addition, you can use features in the AWS WAF Security Automations guide,
to address incoming traffic,
and to customize the design based on your architectural needs.
Now let's go over some of these features in detail
and describe how you can address the control of incoming bot traffic.
The AWS WAF Bot Control rule group helps manage bot activity to your site
by identifying, categorizing
and detecting high confidence signatures of bots.
In the WAF console,
you can see a sampling of how much of your web applications current traffic
is coming from bots.
Although the rule group is fully managed, there may be cases of it blocking legitimate traffic,
also known as false positives.
And you'll want to utilize Scope Down Statements or WAF Labels
to customize it to minimize that.
AWS Bot Control doesn't block bots that are known by AWS
to be common and verifiable bots,
such as minimize these errors by default.
False positives come with almost any web solution,
so you will need to consider a fair trade off between blocking
and allowing illegitimate traffic.
A common attack pattern called a DDoS attack
uses bots to try to overload your application with floods of requests.
Here, you can set up a WAF rate-based rule
that can block requests from these spots after reaching a desired threshold.
In this case, you would want to set a blanket rate base rule
that is two to three times repeat traffic,
so you don't accidentally block your customers.
I do encourage you to dive deeper into your application logs
to determine the traffic behavior
and fine tune multiple rate based rules to reduce the attack surface.
Next, the AWS WAF Security Automation template
is a popular go to CloudFormation template,
to custom features and roles to your WebACL.
Here I want to talk about three different features.
First, we have a feature that blocks reputation lists,
which uses a CloudWatch event to trigger a Lambda function every hour.
This checks third party lists for updated malicious IP ranges
and adds that to the AWS WAF deny rule set.
AWS does provide their own IP reputation role,
and the AWS managerial groups as well,
which is easier to use with just a couple of clicks.
One behavior that scanners and puts into follow
is to generate HTTP firewall on web applications
as they're looking for pages that do not exist on your website.
The second feature of the WAF Security Automation template
cancel its behavior.
Here, the CloudFormation template deploys a Lambda function and Athena resource
to identify specific error codes in your application logs.
It adds the IP address to an IP Block rule
once the error code count reaches a threshold you defined.
You can also customize this to look for other request details
if you're seeing any other bot behavior.
Lastly, the template allows for adding a honeypot to your application
that can be helpful to address pop behavior as well.
A honeypot is a hidden link that is embedded into your application code,
and humans certainly would not click on this.
The way to do this is by leveraging an API Gateway endpoint,
which will then trigger a Lambda function,
and add device IP address to your WAF block list.
Using WAF allows you to block traffic at the edge before it even reaches your application.
Scenario control mechanisms directly in your application
requires you to scale your resources
to match the additional demand which can be unpredictable
and cause potential bottlenecks.
Also, you would need to configure your own mechanisms to detect bots
where AMS has a dedicated threat research team.
Lastly, when testing your WebACL,
we highly recommend setting rules to count instead of blocked for baselining,
so you can customize it further
until you're comfortable with deploying it to production.
WAF can help protect your application at the edge,
but you'll need to understand that security doesn't stop there.
You should scrutinize each layer of your architecture
for additional protective capabilities.
In this episode, we've explored how bot traffic can behave
and how AWS WAF enables security at the edge
before it reaches your application.
Feel free to check out the links below for more details.
Thank you for watching Back to Basics.
See you next time.