NoSQL databases such as Redis have become increasingly popular due to
their focus on very fast query speed and flexibility of schema,
among other reasons.
In this video,
we will look at how a Redis cluster using the Redis on Flash
data storage tier can be deployed on the HPE Apollo 4200 Gen10+ platform.
We'll show how the tremendously dense storage
capacity and processing power of Apollo can produce
a high performance Redis deployment with the
extreme low latency that Redis databases require.
Redis is unique among NoSQL databases, and databases in general,
in that it traditionally has been an all-in-memory application.
Redis databases weren't even allowed to be larger than the available memory
space to ensure that the entire database would always operate completely in RAM.
While this has always made
Redis one of the fastest performing and lowest latency databases,
it also means that Redis databases need to remain relatively small
and almost always need to be sharded across multiple server nodes.
The open source version of Redis
still operates under these restrictions.
However, the commercial version, called Redis Enterprise,
has a relatively new feature
called Redis on Flash that reduces much of the restrictions.
Redis on Flash creates a tier of available storage, only using NVMe drives,
that Redis can use to house less used data within the database.
This lets the current workload still reside completely in-memory
but allows the database to grow larger than the
available RAM without necessarily requiring sharding and multiple hosts.
The HPE Apollo 4200 Gen10+ system
is ideal for applications that require a very dense onboard storage footprint
to both maximize storage capability and minimize latency.
The massive NVMe storage capacity provides an optimal home for
the Redis on Flash tier that we will be highlighting today.
In addition, a single 4200 Gen10+ system
supports a total of 64 CPU cores and four terabytes of RAM per processor.
Perfect for high performance and memory-intensive requirements
for a database app such as Redis.
So let's talk about how we can deploy a Redis on Flash tier
for a Redis cluster using Apollo 4200 Gen10+ systems as the hosts.
We put together a test environment
where a three-node Redis Enterprise cluster
was deployed on three Apollo 4200 Gen10+ units.
We divided the testing into two phases,
one with a single NVMe drive hosting Redis on Flash per host,
and one with a software RAID zero set using three NVMe drives per host.
We ran five different test scenarios for a single Redis database.
We started with a basic database,
no clustering, no replication, no persistence.
We then moved to a cluster database with two shards.
We took that same two-shard cluster and added replication
for a total of four shards.
We repeated the same environment as test three, but we added persistence.
And then in test five, we repeated the same environment as test four,
but we changed the database settings to force the spreading of
the shards across the nodes and make all the shards primary shards.
to do this, we had to turn off replication because the Redis test environment
license that we were using only allows for four shards.
We used the Redis performance test tool
"memtier_benchmark" that comes with Redis software
to populate the database and run loads against it.
We followed the examples listed in the Redis Enterprise
documentation under "Benchmark a Redis on Flash enabled database"
found at the link shown here.
One thing to note about setting up the software RAID set.
Redis has a built-in script in the Redis
install directory called prepare_flash.sh.
This script goes out and finds all the unused NVMe drives
in the host and combines them into one large RAID zero stripe.
Because of the huge NVMe capacity in the Apollo 4200 Gen10+ hosts that we used,
that would have resulted in dozens of NVMe drives included in the set.
Not wanting to allocate all of those drives
just for our Redis deployment,
we manually created the three drive RAID set using the Linux mdadm command.
We duplicated all the settings for the database listed in the benchmarking guide
including setting the database memory limit at 100 gigs and
the RAM to Flash ratio to 30%, or 30 gigs.
This ensures that both RAM and the Flash storage would be used during the test.
The results for the testing were impressive as compared to
the expected performance numbers listed in the benchmarking guide.
The expected numbers were around 160,000 operations
per second without replication, and 115,000 with replication enabled.
In both cases, average latency was expected at below one millisecond.
In these graphs,
the expected numbers are represented in blue,
while the Apollo test numbers are represented in green.
As we look at the graphs, we see that in most instances the
Apollo hosts were able to get Redis to put out significantly better performance
than what should be expected for both throughput,
which you see on the left, and latency, which you see on the right.
Performance significantly improved when sharding is introduced in test two,
which makes sense as that allows more
concurrent client query jobs across the hosts.
However, simply sharding can't account for this level of performance.
That's where the Apollo systems take over.
Adding replication or persistence in tests three and four
didn't cause any performance degradation,
and neither did switching between single disk and the three disk software RAID set.
While we once again see in test one that Redis
just runs better in a sharded environment,
we also see more than double the throughput and less than half of
the latency that Redis would expect in more real world test configurations.
This just points to the superior performance capabilities of
the Apollo hosts across CPU, RAM, and NVMe disks.
Bottom line is, since Redis recommends using sharding
and replication in production environments, and since Apollo shows
almost double or even triple the expected performance
for throughput and a half or a third
the expected latency,
any Redis deployment on Apollo is very likely to outperform commodity
servers on the stats that matter the most to Redis users.
For more information on the Apollo 4200 Gen10+ system,
and the other elements that make up this solution,
please check out the links shown here.
We also recommend checking out the HPE Technology YouTube channel
for other HPE workload solutions for NoSQL users.
Thanks very much for watching.