Hello, and welcome to an introduction and a demo of Amazon RDS Proxy.
In this video, we'll introduce Amazon RDS Proxy
we'll talk about some of its benefits
we'll show the architecture of our demo application
and then we'll spend most of the time demoing setup of a proxy and using it from a Lambda function.
Amazon RDS Proxy is a fully-managed, highly-available database proxy
for Amazon Relational Database Service, or RDS
that makes applications more scalable, more resilient to database failures, and more secure.
Amazon RDS Proxy can be used for serverless applications built in AWS Lambda
but is also a great feature for applications running in containers on ECS or on EC2.
The same advantages are realized for all application types
without having to standup, run, and maintain your own database proxy.
Amazon RDS Proxy sits between your application code and your relational database
and managed the database connections created by your application to make more efficient use of the database's resources.
Three of the main benefits of using Amazon RDS Proxy are scaling, resilience, and security.
Let's talk a bit more about each one.
An Amazon RDS Proxy maintains a pool of established connections to RDS database instances,
reducing the stress on database resources that typically occurs when new connections are established.
RDS Proxy also shares database connections,
so that there are fewer total connections accessing the database.
This connection pooling enables your database to efficiently support a large number of application connections
and frequency of connection establishment,
so your application can scale without comprimising performance.
Amazon RDS Proxy minimizes application disruption from outages affecting the availability of your database,
by automatically connecting to a new database instance while preserving application connections.
When failovers occur, rather than rely on DNS changes to reroute requests
RDS Proxy is able to route requests directly to the new database instance.
This can reduce failover times for Aurora and RDS by up to 66%.
Amazon RDS Proxy is highly-available itself and deployed over multiple availability zones.
Amazon RDS Proxy gives you additional control over data security
by giving you the choice to enforce IAM authentication for database access
to avoid the risky practice of hard coding database credentials into application code.
RDS Proxy also enables you to centrally manage database credentials using AWS Secrets Manager.
Here is the architecture of our demo.
This architecture shows a simple serverless API fronted by API Gateway.
Lambda hosts the application logic of the API which includes running queries against a database,
in this case an Amazon Aurora MySQL database.
Now we'll change the architecture to include Amazon RDS Proxy.
As you can see, the proxy sits between our application code and our Aurora database.
Our application connects to RDS Proxy in the same way it connects to Amazon Aurora
so minimal code changes would be needed to make this update in most applications.
And now, time for the demo.
In this demo we will walk through some existing resources that are setup already
such as the Aurora database, the API, and the Lambda function.
We will then walk through the steps of creating an Amazon RDS Proxy and using it in our Lambda function.
We'll close by showing some test results that illustrate the benefits of using a proxy.
Now we'll head over to the AWS console.
First, let's take a look at the API defined in API Gateway.
It's defined as a simple API with one route
which allows us to pull a list of contact information such as name and email from the database.
Now, we'll look at the database.
It is an Aurora MySQL database with 2 instances
which are deployed across two availability zones.
Here is a quick look at the Lambda function.
The Lambda function is attached to a VPC and implements the application logic behind the API.
This includes querying for a list of contacts from the database
and building the response to return to API Gateway.
We'll dive deeper into the code later in this demo.
Now, let's create a database proxy.
The first step is to create a new database secret in AWS Secrets Manager.
This secret will be used by RDS Proxy to connect to the database
without having to pass credentials in from the code.
First, click on Store a new secret.
This secret is credentials for an RDS Database.
Specify the database username and that user's password.
Leave the default encryption key and then specify the Aurora MySQL database.
Choose the name for the secret.
Click on next.
I am leaving rotation disabled for this demo.
Click on next again.
And finally, click on Store to store the secret.
Now it's time to create the proxy.
To create a proxy we'll go to the RDS console
and click on proxies in the left-hand navigation pane.
You can see there is an existing proxy already.
Let's go through the steps of creating a new one.
Start by clicking on Create Proxy.
Specify a unique ID for the proxy.
I'm selecting Require TLS.
This requires connections to the proxy from the app to be TLS connections.
I'll leave the idle connection timeout at its default value.
This is the time the proxy will allow connections to be idle before closing them.
Choose the Aurora database.
I am leaving the connection pool maximum connections value at the default of 100.
Note that this value is a percentage of the number of maximum connections that the database allows
and not an absolute number of connections.
This value should be set lower if there are other applications or proxies using this database.
Next choose the database secret created earlier.
Leave the role selection as create IAM role.
This will create an IAM role which allows RDS to access the database secret.
In this demo I am selecting require IAM Authentication.
This requires the application to use IAM auth when connecting to the proxy.
This is a nice feature as it means our application will connect using a role
rather than have to manage database credentials in the code.
The list of subnets is populated based on the VPC where the database is deployed.
Here, I'm unchecking the public subnets,
leaving only the private ones.
Under additional configuration we can choose which security group to associate with the proxy.
In this example I am choosing the RDS security group
which happens to be the same security group associated with the database.
Then click Create proxy.
While the proxy is being created, let's talk a bit more about security groups
and how they work with RDS Proxy.
Security Groups are found in the VPC console.
In my case, filtering by RDS shows two security groups.
One is the application security group associated with the Lambda function.
The second is the database security group associated with both the database and the proxy.
Looking at the inbound rules for the database group shows two rules,
both allowing inbound connections on the MySQL port.
One is inbound to the database group from the application group.
The second is inbound to the database group from that same database security group.
To connect to the database through RDS proxy two inbound rules are needed.
First, connections need to be allowed to the proxy from the application.
Second, connections need to be allowed to the database from the proxy.
Both of these are port 3306.
In my case the proxy is in the same security group as the database
so I needed to allow inbound connections to the database group from the same database group.
Just remember you need to allow connections to the proxy from the app
and to the database from the proxy.
Now let's check to see if the proxy is created.
Yes, it is ready.
I'm going to go in and copy the proxy endpoint to use in the lambda function.
Here is the Lambda function.
First, I'll scroll down to the environment variables.
I created these environment variables when I created the Lambda function.
This is things such as database name, username and endpoint are configured.
In this way I can access them in the Lambda without having them directly in the code.
I'll paste in the proxy endpoint.
And click on save to save the variables.
Now I'll configure the execution role for the lambda function.
This is what controls what the Lambda function is allowed to do in AWS.
I'll click on the role and it is opened in the IAM console.
As of now there are two polices attached.
These policies were attached when I created the Lambda function
and they allow the Lambda to do basic things
like log to CloudWatch Logs and attach to the VPC.
What is missing is the ability for the Lambda to connect to the database proxy.
I could add the policy here but there is an easier way.
If I go back to the Lambda function I can attach a proxy to the function.
To do so, click on Add database proxy and choose the proxy that was just created.
By doing this the Lambda console takes care of adding the necessary policy to the role.
I'll go back to the IAM console to verify this.
I can see that there is a new policy attached.
If I open up the policy,
I can see it allows connections to the proxy.
Now I'll walk thorough the Lambda code.
At a high-level, whenever an API request is made the Lambda function is invoked.
In this simple app the Lambda connects to the database,
runs a query, and uses the results to build the API response.
Let's take a look.
First I'm pulling in the AWS SDK as well as a javascript MySQL client.
Since the proxy requires IAM auth, I need to get an authentication token.
This is done by first creating an RDS signer object.
The object is created using parameters from the environment variables
such as region, endpoint, port and user.
The signer is then used to get an auth token.
The database connection object is created using environment variables,
and the password is set to be the auth token.
Now the call is made to connect to the proxy.
This code is the same code as if connecting directly to the database.
After the connection succeeds the query is executed.
In this simple example the query is just a select all rows from our contacts table.
When the query returns the results are used to build an API response to return to API Gateway.
Now I'll test out the Lambda function.
For a quick test I'll click the Test button in the Lambda console.
This is successful and the list of contacts from the database is shown.
Now I'll hit the API from a Postman client.
This request is a simple GET request to the / contacts path.
And we see this works as well.
And there it is.
I created an Amazon RDS Proxy and configured the Lambda function to use the proxy.
Not to worry if you didn't catch it all a code repo for the lambda will be linked from the video.
Now let's take a look at some test results comparing
using Amazon RDS Proxy and connecting directly to the database.
These results were captured when creating the RDS Blog article linked in the video description.
The following graphs show the difference in client recovery times during a database failover.
These results are from an Amazon Aurora MySQL database.
The orange line shows recovery times for connecting directly
while the blue is using Amazon RDS Proxy.
Client recovery times in the test workloads were 51 to 79 percent quicker for Amazon Aurora MySQL.
This graph shows results from a test with a multi-AZ RDS MySQL database.
The yellow and grey lines are results from connecting directly to the database with two different drivers.
The orange and blue lines are the recovery times using the same drivers and connecting through RDS Proxy.
Recoveries were 32 to 38 percent quicker for RDS Multi-AZ.
Be sure to check out the blog post for more details on these results
as well as best practices for client configurations
to reduce recovery times during a failover.
And now we are at the close.
In summary, I gave an intro to Amazon RDS Proxy
and talked about the scaling, resilience, and security benefits.
I created a proxy and showed configuring the Lambda function to use the proxy.
And I showed test results which highlighted potential resilience benefits during database failovers.
For more information, you can go to the landing page shown at the end of the video.
Thanks for your time, I hope you learned more about Amazon RDS Proxy.