- Hey, this is Kirk from the DynamoDB team,
and today we are talking about import from S3
and what you need to know about it.
Import from Amazon S3 for Amazon DynamoDB
is a fully managed feature,
ready to import your data files from S3
into a new DynamoDB table.
There's no code to write, no AWS Lambda functions
to implement and no other services to set up and manage.
For data formats, you can import CSV,
DynamoDB JSON and Amazon Ion formats.
If you're not familiar with DynamoDB JSON
or Amazon Ion formats, I've put links
in the description for more information.
The most common use cases for this feature are
when you need to do a migration from another database,
importing data from S3 for a one-time data load,
to move and create secondary copies
of a table to a different AWS region or account
and to import data that was previously exported
with the DynamoDB export to S3 feature.
When you initiate an import from S3 with this feature,
DynamoDB creates a new table
and any secondary indexes you specified
with enough partitions based
on the uncompressed size of the import files
DynamoDB manages capacity during the import,
so you don't have to, like you might
if you were doing the data load on your own.
When performing an import, you can load one file
or multiple files with the same prefix from an S3 bucket.
Files can be compressed in Zstandard or GZIP format
or can be directly imported in uncompressed form.
The cost of the import from S3 is based
on the uncompressed size of all the files processed
and doesn't consume regular table capacity units
or requests, but more on cost here in a little bit.
Quick tip for you, when creating a new table
as part of the import, pay close attention
to the spelling and case sensitivity of the partition
and sort keys as compared to the files to be imported.
Don't do what I did and mistype the sort key into the form,
run the job and then wonder why the import throws errors.
Complete user error on my part. So learn from my mistake.
Along with double-checking the partition
and sort keys I just talked about,
I also recommend you double and triple-check
your data formats, compression schemes,
file sizes and so on before initiating the import.
This helps minimize the times where
you could get partial data imports, but may still be charged
for the full file size for that import.
When you do need the text of any info
or errors from the import process,
they're available in Amazon CloudWatch.
This includes information about the import,
what errors, if any, were encountered, which items errored
and what actions you might need to take
to resolve any problems.
Let's look at a quick cost example.
Here we compare the cost of writing code
to import data yourself, as opposed to using
the DynamoDB import from S3 feature.
With 100 million items, averaging two kilobytes each
and one global secondary index,
you can see the cost of doing this
with the import for S3 feature is much cheaper.
That's even without factoring in
the time it would take you to write
and test your own code to do the import.
$28.65, as compared to $500, is a big difference.
For more information on cost for import from S3,
check the links in the video description.
Quick cost saving tip for you,
if the new tables will ultimately have secondary indexes,
create those indexes as part of the import
from S3 table create process.
This way, secondary indexes are free to create,
as the cost of doing an import is based
on the import file sizes, not on any capacity used
to create the table or secondary index,
like you would incur if you created
the secondary index at a later point.
All right, it's demo time.
Today, we're in the console on Imports from S3.
We're gonna click on the Import from S3 button.
We're gonna browse S3 and find
our bucket here, in the directory.
I have five JSUN documents.
They're in DynamoDB JSON format,
but they could be in CSV or Amazon Ion format.
I'm not gonna do each one of these individually.
I'm going to do the entire directory of them.
Hit Choose.
I could do here where this bucket comes
from a different account, but I'm not going to.
Here's where you specify the compression
that you have on your files, if you do.
I don't, so I'm not worried about that,
and here's where you look at the other different format.
Click Next, gonna put in My Table3.
I'm gonna put a partition key of PK and a sort key of SK,
and this is because this is what's in the JSUN documents.
Your documents may vary and your partition key names,
but these need to match what's in your JSON documents.
I'm gonna take the defaults,
but you could customize this table, if you wanted to.
This should look very familiar if you've ever created
a DynamoDB table in the console.
Again, I'm not gonna change any of this stuff,
but I could if I wanted to.
Take the default settings. Hit Next.
Here's a summary of what it's gonna do,
and then I click Import,
and now it is off to the races and importing.
As you can see here, the import has completed successfully.
We can refresh this
and see the details of what actually happened.
Here's where we can go
and view the CloudWatch logs, If you want to.
Let's actually go look at the table itself.
It's My Table3.
I'm gonna do Explore items, and now you can see all
of these items are in this table here.
In summary, using the import from S3 feature of DynamoDB
to create new tables is easier
and cheaper than creating your own data import process.
There's nothing to manage or implement
as DynamoDB takes care of the process for you.
Thanks to be on the lookout
for more core concept videos here
and on our Twitter handle @DynamoDB.