Welcome to another episode of 'Back to Basics'.
I'm Cheryl Joseph,
and I am a solutions architect at Amazon Web Services.
So today I want to talk to you regarding some streaming options
to capture item level changes in Amazon DynamoDB.
I want to show you some architecture patterns
to build and event-driven, scalable, and a serverless approach
to capture changes as they're occurring in a DynamoDB table.
So let me first familiarize you with DynamoDB.
Amazon DynamoDB is a fully managed, no SQL database service
that provides fast and predictable performance with seamless scalability.
So similar to other database systems,
DynamoDB stores data and tables.
A table is a collection of data,
and each table contains items.
An item is a group of attributes.
So for example, in a customer table,
each item represents an individual customer,
or in a product table,
each item represents an individual product.
Items in DynamoDB are similar in many ways
to rows, records, or tuples in other database systems.
When you enable DynamoDB Streams in a DynamoDB table,
it writes a stream record,
which contains information
about every modification made to data items in the table.
So here are your clients,
and your clients are inserting an item, a customer, into a DynamoDB table.
When this happens,
DynamoDB makes this change data capture available in a stream.
Next one of your clients updates some customer data,
for example, a phone number.
And when this happens,
DynamoDB will make the before and the after changes
available in a stream.
When it comes to change data capture,
DynamoDB offers you two streaming options
to stream these item level changes.
The first is DynamoDB Streams,
and the other is Amazon Kinesis Data Streams.
DynamoDB Streams has a data retention of up to 24 hours
and currently supports two consumers.
Kinesis Data Streams has a data retention of up to one year.
And with enhanced fan-out,
you can currently have up to 20 consumers.
So depending on your use case,
you can pick either DynamoDB Streams,
or Kinesis Data Streams
to stream item level changes from DynamoDB.
Now let me show you some end-to-end architecture patterns
using streaming from DynamoDB.
So let's take, for example,
you have a new product feature launch of one of your products,
and your customers are providing feedback,
which is stored in a DynamoDB table.
Additionally, you'd like to perform
some sentiment analysis on this feedback in a decoupled manner.
In this scenario,
you can enable DynamoDB Streams
to capture the changes being made in the DynamoDB table.
And with DynamoDB Streams enabled,
you can have the AWS Lambda service,
pull the stream and invoke a Lambda function.
The Lambda function can then call Amazon Comprehend for sentiment analysis
and insert the sentiment response in another DynamoDB table.
You can now use this data to determine the end user impact
of your product feature launch.
So here is another architecture pattern.
You might have a scenario
where you want to deliver item level changes from DynamoDB
to Amazon Aurora or Amazon S3.
In this situation, as your clients update DynamoDB,
your changes are being streamed directly from DynamoDB
to Kinesis Data Streams.
Using Kinesis Data Streams enables you to use different consumers
to capture your changes in DynamoDB.
In this pattern, we have two consumers from Kinesis Data Streams.
The first is a Lambda function
that will update an Aurora database.
You could also send the record from Kinesis Data Streams
to Kinesis Data Firehose.
These changes could then be delivered to S3.
And additionally, you can use Amazon Athena
to query these records.
In this episode, we explored various architecture patterns
to asynchronously capture item level changes
in DynamoDB,
using DynamoDB Streams, Kinesis Data Streams
Lambda, and other AWS services.
Check out the links below for more details, and see you next time.