- [Gy Panday] Leading industry reports
suggest that 60 to 80% of world's data is cold.
Just about every business can benefit
from having an archive strategy
that maximizes the value of cold data.
Modern data archiving is not only about lowering costs
for cold data, but is also about setting up mechanisms
so that when you need to put the data to work,
you can access it as quickly
as your business requirements demand.
Hi, I'm Gy Panday, a product manager for Amazon S3 Glacier,
and I'm excited to show you how you can use S3 Glacier
to improve your data restore time by up to 85%,
enabling you to process restore data faster.
Diana Yeung, a software engineer for S3 Glacier,
will also be joining us in a few minutes to demo
how you can take advantage
of this new S3 Glacier performance improvement.
But before we dive deeper into restores,
I want to give you a quick overview
of Amazon S3 Glacier storage classes.
Amazon S3 Glacier is a leader in cloud cold storage
and was the first to offer a branded solution
in this space over 10 years ago.
Amazon S3 has three purpose-built archive storage classes
for rarely-accessed data.
These storage classes range from milliseconds retrieval
with S3 Glacier Instant Retrieval,
retrieval in minutes to hours
with S3 Glacier Flexible Retrieval,
and retrievals in 12 to 48 hours
with S3 Glacier Deep Archive.
In this short video, we will focus
on the S3 Glacier Flexible Retrieval storage class
and its standard retrieval tier.
Although S3 Glacier Flexible Retrieval
and S3 Glacier Deep Archive require a restore request
before data can be accessed, our customers tell us
that they still want high throughput
in case the need to access large portions of the data.
This is why in November 2022, we increased the rate
at which S3 Glacier accepts restore requests,
effectively increasing the potential restore throughput
for millions to billions of objects by 10 times.
In 2022, customers restored over two billion objects
in just one week and we've seen even bigger workloads
this year as customers take advantage
of the additional restore throughput.
With more and more customers around the world
deriving business value from cold data,
we saw a need to give customers the ability
to not only get higher throughput
but also get restores starting faster.
That's why we are improving S3 Glacier Flexible Retrieval
data restore time by up to 85%.
This performance improvement is available to you
at no additional cost.
Faster data restores automatically apply
to the standard retrieval tier
when using Amazon S3 Batch Operations.
Restores from S3 Glacial Flexible Retrieval
now begin to return objects within minutes,
enabling you to process restore data faster.
With this performance improvement,
you can begin processing your machine learning analytics,
media, and backup data within minutes
of initiating restores.
Now I'm gonna pass it over to Diana
to walk you through a quick demo.
- [Diana Yeung] Hi, everyone.
As Gy mentioned earlier, my name is Diana Yeung
and I'm a senior software engineer for Amazon S3 Glacier.
I've seen firsthand just how important it is
for our customers to have fast access to their archive data,
which is why I'm so excited to be here with you today
discussing the improvement in S3 Glacier data restore time.
I'm going to walk you through how you can use
S3 Batch Operations to initiate standard restores.
We will initiate restores
for 250 objects of 100 megabyte each,
totaling 25 gigabyte that are stored
in S3 Glacier Flexible Retrieval.
Before the demo, I wanna illustrate
just how powerful the new improvements are.
As you can see from the blue line,
the previous restore performance for objects
in Glacier Flexible Retrieval
initiated via the standard retrieval tier,
typically finished within three to five hours.
With the improvements we have made,
the same restores will now start within minutes.
Let's show you how.
Before you start using S3 Batch Operations,
S3 must have permissions to perform
S3 Batch Operations on your behalf.
Grant these permissions through an AWS Identity
and Access Management role, also known as an IAM role.
I've already applied the permissions for this demo.
I've also configured S3 event notifications
to report when restores start and finish.
Now let's go ahead with the demo.
You can access Batch Operations
on the Amazon S3 console homepage.
Once here, we can start configuring our job
by clicking on create job.
On the next page, select the AWS region
in which your bucket is located.
Next, you specify the manifest that lists all the objects
you want to restore, which can be a list created
through an inventory report or your own list of objects.
For this demo, I have prepared a CSV file
that lists objects we want to restore.
This file lists the bucket name in the first columns
and the object keys in the second column.
We will click next to the next page.
On this page, you will choose the operation type,
which in this case is restore.
Once selected, a new set of options appear below.
Since the objects we want to restore
are S3 Glacier Flexible Retrieval,
we will select the first option under restore source
that says Glacier Flexible Retrieval
or Glacier Deep Archive.
With this relatively small restore,
we should easily be able to do what we need
with the restore objects within one day,
so I'm going to select one day retention.
If your data was in the S3 Intelligent Tiering
Archive Access Tier,
you could still benefit from faster restores,
but the console won't ask you to specify
the number of days to store the restored copy
because your objects will be moved
to the S3 Intelligent Tiering Frequent Access Tier.
Finally, since faster restores
through S3 Batch Operations are available
on the standard retrieval tier,
we will select that option and click next.
On the final page, we will configure two more options.
First, we select the destination
to write the completion report.
Then we choose the IAM role that gives S3 Batch Operations
the permission to perform this operation.
When we click next, it takes us to the review page.
We review details, and once everything looks correct,
we will click create job at the bottom.
Now our S3 Batch Operations restore job will be created.
Let's wait for the job to be ready.
I will refresh the page and it looks like it is now ready.
Let's select the job and click run job at the top.
Let's review the details one more time,
and once everything is correct,
you can confirm by clicking run job at the bottom.
We'll come back shortly when the job finishes.
The job just finished.
You can see here that all the objects
we initiated at time 7:14.
The first object was restored 22 minutes
after we initiated the restore,
and the last object was restored after 26 minutes.
So in summary, we restored 250 objects totaling 25 gigabyte
in 26 minutes from S3 Glacier Flexible Retrieval.
I'm really excited for you to start using this new feature
and I look forward to hearing about even more ways
you are reinventing your business with archive data.
Back to you, Gy.
- [Gy Panday] Thank you, Diana.
You just saw an example of a workload
that can finish in under an hour.
Now I want to discuss how you can benefit
from faster retrievals with larger jobs.
Let's walk through an example.
Say you need to generate thumbnails
of images stored in S3 Glacier Flexible Retrieval.
You have 10 million of these images,
totaling around 95 terabytes.
In the first step, you specify all your objects
in an S3 Batch Operations job and initiate your restore job
using the standard retrieval tier.
Your restores automatically start within minutes.
Now you could invoke AWS Lambda functions
across all your objects after all the restores complete,
but because restores are starting sooner,
you can use that to your advantage
to complete the overall workflow faster
so you can configure Lambda to run
based on restore event notifications,
which can run Lambda function to create a thumbnail
as soon as each object completes restoring.
This processes the data in parallel
to your completing restore job.
For large workloads, this can significantly reduce the time
it takes to process data end to end.
Now, let's summarize the benefits
of this S3 Glacier performance improvement.
Faster restore speeds on the standard tier
are now available using S3 Batch Operations.
Even your large workloads can now benefit
from higher throughput available
from restores starting in minutes.
Many customers tell us that they're keeping the data
because they recognize its future potential.
To learn even more about how you can maximize the value
of your cold storage with Amazon S3 Glacier,
scan the QR code to access our new ebook.
This ebook will tell you why and show you how
Amazon S3 Glacier is helping organizations of all sizes
and from all industries transform their data archiving
to unlock business value, increase agility,
and save on storage costs.
Maximizing the value of archived assets
to reinvent your business and accelerate your innovation,
that is the real value that Amazon SG Glacier delivers.
We are excited to see how you use faster data restores
with S3 Glacier to further modernize your data archive.
Thank you for your time.