RACHEL DEACON SMITH: Hi.
In this video, I'll show you how you
can replicate data from an Oracle database
into BigQuery using Cloud Data Fusion's replication
accelerator.
My name's Rachel Deacon Smith, and I'm a developer advocate
at Google Cloud.
If you're an ETL developer or a data analyst,
you may want to continuously replicate your operational data
into BigQuery in real time, so you can make timely,
data driven business decisions.
Data Fusions, with a driven accelerator,
integrates with Data Stream, Google Cloud's
serverless and easy to use, change data capture service.
This end-to-end Cloud native solution
will fuel your analytic environment
with instant insights, removing the need
to develop bespoke and high maintenance solutions.
So let's jump into the demo.
Creating a new replication job with Cloud Data Fusion
is super easy.
You first want to launch Data Fusion from Cloud console,
and you can find this under the Big Data section.
If you don't have a Cloud Data Fusion instance yet,
you'll need to create one.
So you'll need to specify some basic parameters
such as the instance name and the region,
and you'll also need to add the replication accelerator.
You'll also need to make sure that both the Data
Fusion and Dataproc service accounts
have the Data Stream admin role assigned to them,
and you can do this from the [INAUDIBLE] page
in the Google Cloud console.
OK, so once your instance is ready,
you're now ready to configure your Oracle
to BigQuery replication.
So we'll just click on the tile named Replicate,
and then on the big green button selecting
Create a replication job.
Setting up the replication is super easy,
because we have a wizard driven UI that will guide you
through the entire process.
So you just need to provide a name and a description,
and then you can select the source database type.
So while you can replicate from MySQL or SQL server,
we're interested in replicating from an Oracle source,
so we'll put Oracle by Datastream.
And then we need to provide the source connection details,
so we'll need to enter the region.
We need to provide the hostname and the port.
We need to give a system identifier
and add any credentials that we need
to connect to our database.
We also want to bring with us a historical backfill,
so we'll leave the Replicate Existing Data setting enabled.
Next, you want to pick the source objects,
so these are the tables in the columns
that you want to replicate.
So for example, in a given table,
we can choose whether to replicate all columns
or to select a subset of columns that we want to replicate.
We can also choose the type of events to replicate,
such as inserts, updates, and deletes.
Once we're happy with our selection,
we can just click Next to configure the target.
Since Cloud Data Fusion is fully integrated
with the rest of Google Cloud's ecosystem,
configuring the BigQuery target is pretty simple.
You just provide some basic information,
such as the BigQuery data set name, and you're ready to go.
You can actually leave the data set
details blank to default to the name of the source database
or schema.
So we can click Next to configure now
the advanced properties.
So if you have a good idea on the amount of data
that you're planning to replicate,
you can tweak the replication configuration here.
So for example, how much data you
expect to replicate in an hour.
I'm happy with this as it is, so I won't make any changes.
Once the source and target details are provided,
the replication wizard will automatically
perform a pre-deployment assessment
and proactively advise you on any potential issues prior
to the replication deployment.
So for example, it can point out any schema compatibility issues
between the source and the target, and in this case,
the assessment has indicated that we're missing a feature.
So by clicking on the missing features link, I can drill down
into the issue, and in this case,
realize that I'm missing a primary key.
So once I fix my problems in the source,
I can just click the Refresh button on the top right corner
and move on to the Summary page.
After reviewing all the parameters provided,
all that's left to do is click the Deploy Replication
Job to start your job.
And once the replication is running,
monitoring is super simple.
You can natively monitor the replication progress,
the errors, and the throughput and latency
via a centralized and easy to use UI.
And now once your data is in BigQuery,
you'll be ready to run real-time analytics
or build a robust reporting platform by connecting
to business intelligence services,
like Looker, to build powerful, real-time business dashboards
ultimately allowing you to take immediate actions based
on streaming data.
So that's it.
I've quickly shown you how to easily
replicate data from an Oracle database
into BigQuery using a wizard driven Cloud Data Fusion
accelerator that is integrated with Google
Cloud serverless and easy to use CDC Offering I
hope you've enjoyed it.
So you can get started from the Google Cloud console or learn
more at cloud.google.com/data-fusion.
Thanks so much for watching.