PRIYANKA VERGADIA: Are you migrating
your monolithic application to microservices on GKE?
Then I bet that data is a huge part of that migration.
Stay tuned to find out more in today's episode
of Get Cooking in Cloud.
[MUSIC PLAYING]
CARTER MORGAN: When it comes time to migrate data,
the strategy you use depends on the answers
to a few important questions.
PRIYANKA VERGADIA: Well, how much
data do you need to migrate?
CARTER MORGAN: How often does this data change?
PRIYANKA VERGADIA: Can you afford the downtime
represented by the cut-over window
while migrating that data?
CARTER MORGAN: What is your current data consistency model?
There's really no single best approach to migrating data.
Today, we will share four different data migration
approaches, each of which tackles different issues
depending on the scale and the requirements of the data
migration.
PRIYANKA VERGADIA: The data migration approach
you use is based on the cut-over window size
that your application can tolerate,
the refactoring effort that your team can handle,
and the amount of flexibility each approach offers.
CARTER MORGAN: The scheduled maintenance period
is ideal if your workloads can afford a cut-over window.
It's scheduled in the sense that you can plan when
the cut-over window occurs.
Here's how it looks.
Copy data from legacy sites to new site
in order to minimize the cut-over window.
For more details, refer to the earlier episode
on migrating in stages.
Compare the legacy data against the new data
set to make sure there weren't any copy errors.
Stop further data from being read into the system
during the migration.
Synchronize any changes that happen after the initial copy.
Deploy new versions of the workloads and services
that use a new site.
Start the workloads and services.
Retire the legacy site when you no longer need it
as a fallback option.
This approach places most of the burden on the operations side,
because minimal refactoring of the workload and system
is needed.
PRIYANKA VERGADIA: Not all workloads
can afford a long cut-over window,
so the continuous replication approach
builds on the scheduled maintenance approach
by providing a continuous replication
mechanism after the initial copy and validation steps.
Here's what that would look like.
Copy data from legacy site to the new site
in order to minimize the cut-over window,
just like we did in the scheduled maintenance approach.
Then, we compare the legacy data against the new data
to make sure there weren't any copy errors.
Set up a continuous replication mechanism from the legacy site
to the new site.
Stop further data from being written to the system
during the migration.
Refactor workloads and services to use the new site
and wait for the replication to fully synchronize the new site
with the legacy site.
Then, start the workloads and services.
Retire the legacy site when you no longer need it
as a fallback option.
This approach is more complex than the scheduled maintenance
approach, but it minimizes the time
for the required cut-over window by minimizing
the amount of data you need to synchronize.
CARTER MORGAN: If your workloads have hard,
high availability requirements and you can't afford a cut-over
window, a different approach-- what we're calling the Y
recipe--
would make sense.
In this recipe, the workload is writing or reading
data in both the legacy site and the new site
during the migration.
Here's how it looks.
Refactor subsystems to write data
both to the legacy and the new site
while reading from the legacy site.
Copy data into the new system that
was written before the new site began to write data.
Compare the legacy data against the new data
set to make sure there weren't any errors.
Switch read operations from the legacy site to the new site.
Perform another round of data validation and consistency
to check data.
Disable writing in the legacy site,
and retire the legacy site when you no longer need it
as a fallback option.
Unlike the scheduled maintenance and continuous replication
approaches, the Y writing and reading recipe
shifts most of the effort from the operations side
to the development side due to the multiple refactorings,
but it does not require a cut-over window,
which is a plus.
PRIYANKA VERGADIA: Now, if you want
to reduce the large amount of refactoring effort needed
in the Y approach, you can centralize
the data read and write operations
by refactoring the workloads and services to use data access
microservice.
This scalable microservice becomes the only entry point
to your data storage there, and it
acts as a proxy for that layer.
It's actually pretty similar to the Y approach
we discussed earlier.
The only difference is instead of refactoring
the entire service, we focus the refactoring effort
on the data access microservice alone.
Refactor the data access microservice
to write data both in the legacy site and the new site.
Reads are performed against the legacy site.
Identify the data that was written
before you enabled the writes in the new site
and copy it from the legacy site to the new site.
Along with the refactoring above,
this ensures that the data stores are aligned.
Perform data validation and consistency checks,
comparing data in the legacy site
against data in the new site.
Refactor the data access microservice
to read from the new site.
Perform another round of data validation and consistency
checks comparing data in the legacy site
against data in the new site.
Refactor the data access microservice to write only
in the legacy site.
When you no longer need the legacy site
as the fallback option anymore, retire it.
In a microservice architecture, this recipe
is the preferred method and gives you
the maximum flexibility, because you can refactor this data
access component without impacting
other components of the architecture
and without requiring a cut-over window.
It does put a burden on the developer side,
but it's significantly less than the Y approach we discussed.
CARTER MORGAN: So in summary, we looked at four recipes
to migrate data, although picking the right option
would depend on weighing the pros
and cons for your situation.
For more information about migrating a monolith to GKE
and these data migration strategies,
refer to the article linked in the description below written
by our friends, [? Marco ?] [? and ?] [? Teo. ?]
PRIYANKA VERGADIA: Well, there you have it.
If you are looking to migrate a monolithic application
to microservices, then today, you've
got a taste of different data migration recipes.
CARTER MORGAN: And don't forget to check out
the collab in the link below to experience
an end-to-end migration of an e-commerce website.
PRIYANKA VERGADIA: That's all for today
on Get Cooking in Cloud.
Let us know how you're liking the series in the Comments
section, and hit those Like and Subscribe buttons for more
cool Google Cloud content.
[MUSIC PLAYING]