- Hi, I'm Michael Soo,
I build migration services here at AWS,
and I wanna share some information with you
about the tools that we offer customers
to migrate their data warehouses into the cloud.
A data warehouse is really just a database
that's been optimized for analytics,
and that means managing big data to compute business metrics
or to feed advanced analytics, like machine learning models.
So, Amazon Redshift is our managed data warehouse offering.
With Amazon Redshift,
you let AWS automate all the day-to-day administration tasks
for your data warehouse,
and it lets you leverage the compute and storage resources
that we can provide at cloud scale.
So, tens of thousands of customers use Amazon Redshift
to gain new insights from their data every day.
And you can use Redshift
to query, literally, exabytes of data
across your data warehouse, your operational data stores,
and your data lake using standard SQL.
You can integrate other AWS services
such as Amazon EMR, Amazon Athena,
Amazon SageMaker, AWS Glue, AWS Lake Formation,
and Amazon Kinesis
to utilize all the capabilities that we provide
in the AWS Cloud.
So, how can you migrate into Amazon Redshift?
Well, if you've been operating your data warehouse
on premises,
it can be a significant investment to migrate into AWS.
For example, you may be using a proprietary engine
that offers a feature set
that's very different from Amazon Redshift,
and that would mean lots of data conversion,
code refactoring, and maybe even re-architecture
of your data ingestion and consumption applications.
But my team has been working very hard
for the last few years
to automate many aspects of this migration process,
so we've really listened to our customers
and delivered dozens of features in our tools:
the AWS Schema Conversion Tool, or SCT,
the AWS database migration service, we call it DMS,
and Amazon Redshift to accelerate your migrations.
So, what I'd like to do is maybe share a couple of examples.
Your workload may rely on non-standard, proprietary features
that aren't directly supported by Amazon Redshift.
So, for example, you could be using set tables.
And set tables are very similar to primary keys,
except that they guarantee what we call full row uniqueness.
And there are some subtle differences,
but it's essentially as if all the attributes in the table
were part of the primary key.
So if you have applications that depend on set semantics,
you need a way to emulate that same behavior
in Amazon Redshift.
Well, now, we're happy to provide an ability to use AW SET
to automatically convert your SQL code.
SET will analyze your SQL
and inject code into your SQL stream
that will check for duplicates
before they're inserted into the database.
And if you're building a new application
that uses set tables,
we've also published a how-to blog
that describes the transformation
so that you can replicate the process yourself.
So, AWS DMS,
or just DMS,
is our managed database migration service.
You can use DMS to replicate data into Amazon Redshift.
And we've recently enhanced DMS
with parallel-load capabilities
that are optimized specifically for Redshift.
So, in the traditional process, you have a pipeline,
and each step in that pipeline was serial,
so you would first extract data, upload that data to S3,
and then have that data copied into Amazon Redshift.
Well, DMS now supports what we call parallel full load,
and with parallel full load,
the ingestion process into Amazon Redshift
runs at the parallel degree you choose.
You control how many parallel sessions to use
with a DMS task setting,
and you can dial it up or dial it down
to meet your particular use case.
Finally, let me share an enhancement to Amazon Redshift
that we built to make it easier to migrate your ETL streams.
Many customers depend on database scripting to run ETL,
and in a large warehouse,
you may have pipelines with tens of thousands of scripts
that were written in your particular vendor's
proprietary language.
So to help with those migrations,
we created a tool called RSQL.
RSQL is a purpose-built scripting tool
optimized for Amazon Redshift.
With RSQL, you can execute SQL statements to build,
cleanse, and transform your data.
RSQL also includes control logic,
so you can monitor and detect
when your pipeline is executing correctly or not
and then take appropriate action.
So, these are just a few
of the dozens of enhancements we've made recently
to enable migrations into Amazon Redshift.
For more details, you can visit the AWS Blog's website.
See the description for links to relevant posts.
Otherwise, I hope you found this information useful,
and happy migrating.
(upbeat music)