(keys clicking)
(air whooshing) (pencil scratching)
(bell dings)
(engine rumbles)
(bright music) (gear clicks)
- Hello, and welcome to another ACG Project.
My name is David Thomas.
I'm a training architect with A Cloud Guru.
ACG Projects are self-contained activities designed
to be completed in a weekend or less.
These projects give you the opportunity
to get hands-on experience and learn something new.
In this ACG Project,
we're going to improve the performance
of a struggling Python application
by using Amazon ElastiCache for Redis.
This project is my response
to the #CloudGuruChallenge I hosted
in June and July of 2021.
You can find all the details,
including the code repository,
in the challenge blog post linked below.
There are many ways to accomplish this challenge,
and this project just demonstrates how I did it.
So let's get started.
Hello, and welcome to this ACG Project,
improving application performance using Amazon ElastiCache.
In this project, we'll start with a simple application
that queries a database.
As we will see,
this application is performing rather slowly.
To resolve this, we will first deploy
an Amazon ElastiCache for Redis instance.
Next, we will alter our application to check the cache
before querying the database.
If this check fails,
the application will then query the database
and update the cache with the result.
There are three tasks required to complete this project.
First, we'll need to deploy
an RDS Postgres database instance.
Once that's done, we need to connect to it,
create a database and user for our project.
Next, we'll need to clone the application code
from the GitHub repository.
With this done, we can configure the database connection
and start our application.
With the application up and running,
we can now deploy an Amazon ElastiCache for Redis instance
and modify the application to use it.
So now let's get connected to the AWS Console
and deploy our RDS instance.
First, we'll need to deploy an EC2 node
to run our application on.
Clicking on Launch instances.
We can go with the Amazon Linux.
The instance type, we'll stick with the free tier.
We'll use the default VPC,
making sure to set Auto-assign Public IP to Enable.
The default storage should be plenty for our application.
We will create a new security group,
and we'll call this App Security Group.
Set the one rule for this group
to allow SSH from my IP address only.
We will need to create a new key pair.
We'll call our key pair project.
Downloading it, we can launch the instance.
While the EC2 instance is launching,
we can go ahead and deploy our RDS instance.
Clicking on Create database,
we'll choose the PostgreSQL engine.
And we'll choose the free tier template.
We'll call our database project, and we'll set a password.
We'll also go ahead and create a new security group
called DBSecurityGroup.
And we'll click Create database.
While that database instance is starting up,
we can move back to the EC2 console
and see if our EC2 instance is now ready.
With our instance now in the running state,
we should be able to connect.
We can copy the public IP address.
Switching over to our console,
we can connect to the EC2 instance via SSH
using the key pair we downloaded before.
As you can see, we're now connected.
We will need to install a few operating system packages
using the yum installer.
Then we'll install the Python modules
using the pip installer.
We'll install the PostgreSQL dev, the GCC,
and the Python 3 development packages.
With this complete, we can use the pip installer
to install the Python modules.
We'll be installing the Psycopg module,
the Redis module, and the Flask module.
With this done,
we're now ready to confirm the connection to our database.
Switching back to our console,
we switch to the RDS management console.
Click in on DB Instances.
We see our project instance is now available.
From here, we'll copy the endpoint name.
We can save the host name
in an environment variable called PGHOST.
This will make connecting easier,
as the PSQL client will default to using this host.
We'll specify Postgres as a username
and use the default host from the environment variable.
Type in our password.
You can see that we are now connected
to the Postgres server.
While here, we do need to create a database for our project
as well as a user
and grant that user access to the database.
With that done, we can access this connection
and this time connect as the project user.
Now you can see we're connected as the project user
to the project database.
We're now ready to check out the application code,
but first we need to install the git client.
We can now download our code using the git clone command.
To configure an application,
we first need to run the install.sql command.
With that complete,
we're now ready to edit our configuration file.
We'll first need to copy the host name
from our environment variable.
And we'll set the database, the username,
and the password.
With that done, we should be ready to start our application.
Typically, this application will be running
behind a proxying web server.
But for this proof of concept,
I'll just use SSH port forwarding.
We will need to exit our existing SSH connection
by pressing Control + D.
We'll specify port 5000 as the local port
and the loopback address, port 5000,
as the address to proxy.
This will allow us to connect to port 5000
on our work station
and be connected to port 5000 on our EC2 node.
We can change back into our source code directory
and run the application.
Now switch back to our web browser.
Open a new tab.
And access localhost 5000.
After a delay in loading,
we can see the database version and host name are printed.
The elapsed time of five seconds shows
that this application is performing very slowly.
With the application up and running,
we're now ready to deploy our ElastiCache Redis instance.
Switching back to the AWS Management Console,
we'll choose the ElastiCache service.
Clicking on Get Started Now, we can create a Redis cluster.
We'll give it a name of project.
Leave it two replicas.
We will change the node type
to a T3 medium,
as we shouldn't need very large instances
for this proof of concept.
We will create a new subnet group,
calling it the cache subnet group.
And we'll select all available subnets.
We'll leave the default security group now,
but we will change that in a moment.
We leave everything else the default and click Create.
While our Redis cluster is being deployed,
let's go create a security group for it.
Clicking on the VPC service.
We can choose Security Groups.
And choose Create security group.
We'll call this the CacheSecurityGroup.
And we'll add an inbound rule.
We'll choose a custom TCP rule with port 6379,
as that's the default Redis port.
We'll choose the source as our application security group.
And click Create security group.
Now go back to our ElastiCache service
to see how our instance is doing.
As you can see, it's still in the status creating.
This can take some time to complete,
so I'll pause the recording here until it's ready.
There, now that our Redis cluster is
in the available status,
we can go ahead and modify it to use our new security group.
We'll choose our cache security group.
Click Save, and then click Modify.
While that's modifying,
we can go ahead and copy our primary endpoint
and return to our EC2 node.
We'll go ahead and add the Redis URL to our database ini,
as we'll use that to connect from the application later.
We'll create a new Redis section
with one parameter, redis_url,
which equals redis:// then our endpoint name.
Oh, make sure we spell Redis correctly.
And we'll save and quit that.
We first should confirm we can connect to the Redis cluster
from our EC2 node.
We can do that using a few simple Python commands.
We'll start the Python interpreter.
Import the Redis module.
Create a new cache module
using the Redis.from_url parameter.
The ping method returns true,
showing we can connect to the Redis instance.
So now we're ready to our alter our application
to use our new Redis cache.
Opening our application, move to the fetch method,
which actually connects to the database and runs our query.
We'll first set a TTL of 10 seconds.
This is the time to live
or how long our cached results will remain valid.
Then we'll first try to load the Redis section
of our configuration.
We'll save the database results
using the SQL command as the key.
We'll try to retrieve a cached response
using the get method.
If the return value from the cache
is anything other than none, we'll return that.
Otherwise, we'll go ahead and connect to the database.
Connecting to the database
and fetching the response remains much the same.
With our database connection closed,
we can now add the results to the Redis cache.
Before exiting the file,
we also need to import the Redis module.
We can now run the application
and access it from a web browser.
You note that the first access was still very slow.
Refreshing the page, it returns almost instantly,
as we are now getting the cached response.
We did it.
That was quite an impressive performance improvement.
Five seconds may not seem like a long time,
but when you're waiting for a webpage to load,
it can feel like an eternity.
While this application was artificially slowed,
this project still represents a valid way
to cache database responses.
I hope you found this project useful
and learned something new.
Until next time, goodbye.
(inspiring music) (gear clicking)