- [Narrator] In this video,
you'll see how you can perform one-time exports
of third-party data sets from AWS Data Exchange.
To suit your use case,
you can download single file assets to your local computer
or export specific data revisions or entire data sets
from your third-party data subscriptions
to an Amazon Simple Storage Service, or Amazon S3 bucket.
Let's start by reviewing
the AWS Identity and Access Management, or AWS IAM role,
we've prepared for the exports.
You can see that the IAM role includes
an AWS Data Exchange subscriber full access policy,
which permits us to export data from AWS Data Exchange.
We're also going to use AWS CloudShell
to export one or more data sets,
which is why this additional policy
is associated to the role.
We also included an inline policy
that grants Put and List permissions
for the Amazon S3 bucket,
to which we will be exporting the data.
Let's assume this role and continue.
Now let's go to AWS Data Exchange
to find the data set we want to export.
We are already subscribed to this product.
This product contains two data sets,
one of which contains data
that is stored in the London region
and delivered as an Amazon S3 object.
Let's take a look.
This data set has two revisions,
one from July 7th and another from July 29th.
One way we can export entitled data
from AWS Data Exchange is to download single file assets
from a data set directly to a local computer.
To do this, we'll simply select the asset
and then choose the Export action
to Download selected assets.
Now that the download job is complete,
we can open and view the file.
Now let's go to the Amazon S3 bucket we created
for storing data exported from AWS Data Exchange.
You can see that our AWS Amazon Data Exchange
one-time export bucket has been created
in the EU or London region.
Let's go to AWS Data Exchange
to export the data from our subscription.
First, let's see how to export individual revisions
in the data set.
Let's start by using the console
to export the revision created on July 7th to Amazon S3.
To export the revision, we have two options.
The Advanced option lets us specify the naming pattern
for exported assets.
For now, we'll use the Simple export option.
Let's choose our destination bucket.
For this example, we'll leave encryption as None.
Let's export the revision.
As you can see, a new export job has been created
and it's in a Processing state.
Now that the export is complete,
let's review the bucket's contents.
We'll do a refresh.
We can see that the CSV file
from the revision has been successfully exported.
Let's return to the data set
and export the July 29th revision.
This time we'll use the Advanced export option.
We'll select the same Amazon S3 bucket
and then apply a key naming pattern
for the folder naming convention.
We'll use a dynamic reference
that includes the year, month, revision ID, and asset name.
Once we have specified a valid key naming pattern,
the top three assets in the revision are displayed
to acknowledge that our syntax is correct.
Since this revision has one asset,
we are able to see only one entry.
Again, we will leave encryption as None
and export the revision.
Now that the export job is complete,
let's check the objects in our bucket.
The new folder contains year, month,
and revision ID subfolders,
as well as the revision asset, which is an Excel file.
Now let's see how to export the entire data set at once
to an Amazon S3 bucket using a Python script.
To do that, we'll launch AWS CloudShell.
Next, we'll run a command
to clone the GitHub repo script we'll be using.
Once the clone is complete,
we'll grant Execute permissions to the Python script.
Next, we'll update the Python script
to include our bucket name, the data set ID,
the region where the data set is located,
and the same key pattern we used earlier
but with a different folder named one-time-export.
Let's execute the command.
Since the data set has two revisions,
two export jobs are in progress.
The export has completed.
Let's review our bucket.
The one-time-export folder we just created has been added.
The data set revisions were successfully exported
as two separate objects.
You've just seen how you can perform one-time exports
of your subscribed third-party data sets
from AWS Data Exchange.
You can learn more about this topic
in the description and links for this video.
Thanks for watching. Now it's your turn to try.