- [Harshida] Hello, hope you're doing well.
Welcome to the demo on AWS Data Exchange
for Amazon Redshift,
how you can combine third-party dataset with existing data
on Amazon Redshift.
I'm Harshida Patel,
a senior analytics specialist solution architect at AWS.
Redshift's RA3 instance types
provides you separation of compute and storage.
With the separation of compute and storage,
you can size your cluster based on the compute needs
for your workload,
and you only pay for the storage that you consume.
The separation of storage and compute
is the foundation for data sharing.
Now let's cover some of the concepts and terminology
about data sharing.
You have a producer cluster,
which consists of compute and Redshift managed storage.
The producer cluster creates a data share.
The data share can consist of schema or schemas
tables, views, materialized views,
and sequel user defined functions.
When you have consumer cluster,
or you could have consumer clusters,
the consumer assess the data share, it applies security.
And this provides read only access to the shared data share.
When the producer makes any changes to the data,
this provides the consumer
live, transactional, consistent view of the data.
When the consumer is running the query
using the data shared objects,
it uses the compute capacity of the consumer cluster.
The consumer can also join this data,
which is from the data share
to the objects which are locally on the consumer cluster.
The producer pays for the Amazon Redshift managed storage
of the producer cluster,
and the consumer pays for the cluster on the consumer side.
This provides workload isolation.
Data sharing can be enabled within AWS account,
across AWS accounts,
and data sharing across AWS regions
is currently in public review.
Clusters that needs to do data sharing across AWS accounts
and across AWS region needs to be encrypted.
There are different use cases for data sharing.
It could be workload isolation,
or if you want to do collaboration
within different departments,
or within your organization,
or you want to monetize your data
by providing that data as a service to external customers
or external parties.
So, with Amazon Redshift's integration
with AWS Data Exchange,
it enables you to simplify how you can license your data,
which is on the Redshift cluster.
When you create a data share,
this is called as AWS Data Exchange
for Amazon Redshift data share.
The subscriber can subscribe to this data
from the product catalog in AWS Data Exchange.
And as soon as the subscription starts,
they have the ability to analyze this data
till the end of the term of the subscription.
All the billing as well as payment
is all centrally managed.
In this demo,
we'll see how we can combine third-party dataset
with existing data on Amazon Redshift.
The first step is to log onto AWS account.
We'll go to the AWS Data Exchange console,
browse through the product that we are interested in.
In this particular demo,
I'm going to use FactSet supply chain data set product.
We subscribe to the product
by agreeing to pricing terms and conditions.
Then we are going to accept,
or we are going to browse through
the AWS Data Exchange data share.
And from the Redshift cluster,
we are going to create a database for that data share.
Once we have the database available on Amazon Redshift,
we can directly start analyzing that data set.
Let's dive into the demo.
We log on to AWS management console.
Under services, we'll search for AWS Data Exchange.
Under the list of the services,
we'll select AWS Data Exchange.
This will take us to the landing page of AWS Data Exchange.
From the left hand side,
we can browse through the catalog.
And this is the product catalog.
The product catalog consists of data set from third party
for different verticals, automotive, financial services,
or if there are datasets that you're interested
for healthcare and life sciences,
or for media and entertainment.
You'll also see the list of the vendors
from which this third party data sets are provided
on AWS Data Exchange.
And the source of this data provided could be on Amazon S3.
It could be Amazon Redshift or API.
So, for our demo, we'll select Amazon Redshift.
I will search for FactSet.
And we'll select the product that we are interested in.
So, I'll walk you through the steps of how to subscribe
because I'm already pre subscribed
for the supply chain datasets.
So, let's click on FactSet Global Prices.
This contains the information
on what this data set is about.
How is this product offered?
Is it publicly offered or what are the terms and conditions
or subscriptions that you can subscribe to?
We look at the dataset,
and this data set is from Amazon Redshift data share
and the support.
So, all providers will have their email address.
And if you have any questions,
how to reach out to for support,
we'll continue to subscribe to the data share.
And this is where you can provide
the information of your account
before you click on a subscription or subscribe.
I already subscribed to the FactSet supply chain dataset.
And this is where it's already subscribed.
So, now once this dataset that we have subscribed to,
you can look at the definition of the dataset
and this data set is supply chain relationship data,
is built to expose business relationship interconnections
among companies globally.
So, now that we have this subscription,
we are going to go to the Redshift console,
on the left-hand side, you will see data shares.
You can see the data shares within your account,
from other accounts.
The data set that we are interested is from subscriptions.
So, let's click on Subscription
and we are able to see the FactSet data share
for supply chain relationship.
We are going to go to the second step
where we are going to create a database
on Amazon Redshift cluster from this particular data share.
So, this is the subscribed demo Redshift cluster.
We'll click on Datashares
and then select the data share from the subscriptions.
So, this is going to list out
all the dataset shares that are available from subscription
within the account.
And we are able to see
the FactSet supply chain relationship data share.
We are going to click on that.
And then through the console,
you're going to create a database.
And for this database,
we are going to name it supply chain third party,
will be the name of the database.
All of the steps that we have done
going to the data exchange through the console,
as well as creating this database can all be scripted
using API.
So, we'll create this database.
And once we have this database,
now we will log on to Redshift cluster
using query editor version 2.
We are going to go connect to the subscriber demo cluster.
Once we have connected,
we will be able to see supply chain third party
on the navigation tree.
Let's minimize the window
and let's expand the supply chain dataset.
We are able to see three schemas,
and within each of the schema,
if there are tables, views, materialized views,
which are accessed or shared,
we will be able to browse through.
And if you want to look at the table definition,
click on the name of the table,
we can see what are the columns corresponding data type.
So, let's run a query now directly using that data share
from within your Redshift cluster.
So, this is where we are able to,
in three steps, able to access third-party dataset.
You can also join this data to your local data.
And thank you for watching.