[MUSIC PLAYING]
NISHANT KOHLI: Hello, everyone, and welcome to our session
on What's the News.
Today, we'll be covering our dual region enhancement.
In the next few minutes, we'll talk
about our latest dual region enhancements for Cloud Storage.
You'll get an idea of best fit for dual region.
We'll also demonstrate this functionality at the end.
So stay tuned with us.
I am Nishant Kohli, an outbound product manager
for Google Cloud Storage.
And today, joining me is Kumar.
Kumar?
KUMAR NACHIKETA: Hi, everyone.
My name is Kumar Nachiketa, and I
am a storage practice lead for Asia-Pacific at Google Cloud.
Nishant, before we get into the details,
I wanted us to set some context for our customers.
My question to you is, what are the typical problems
that customers face while planning
for georedundancy for object storage,
and what solution goals we at Google Cloud
have in mind to help our customers
with such requirements?
NISHANT KOHLI: Sure, Kumar.
Based on our experience of working
with many of our customers, when they put their critical data
on our object store-- whether they need to distribute it
across a geo for data protection or availability--
there are three typical challenges that come to mind.
Is the configuration setup itself too complicated?
How do I manage for outages?
Will there be complex run books to follow for failover
and fail back?
Also, will there be any challenges
in maintaining continuous access to the data
in the event of a regional failure
or having temporary disruptions.
And simply put, our goal is to ensure we resolve all
of these customer challenges.
We keep it simple and easy for customers to configure and let
us manage the complexity at the back.
Also, despite having our data spread
across multiple locations, we want
to make sure it's truly distributed and maintained
in a single namespace.
So maintaining strong consistency
is very, very critical for us-- for our customers as well.
Also, in any unfortunate instance, disaster,
where region failures do happen, we
wanted customers to seamlessly be able to failover and fail
back, so reducing the customer effort down to zero.
KUMAR NACHIKETA: Wow.
So I'm sure this could bring a great peace of mind
to our customers who are looking for georedundancy
in their object storage, right?
So let's get into the details now.
Help us understand the different location strategies
with Cloud Storage.
NISHANT KOHLI: Sure.
So when you create a bucket, you also
have an ability to choose one of the location
types of your bucket.
There are three type of location types here.
We have regional, dual region, or multi-region.
Now starting with regional, your data
is stored in a specific region of your choice--
for example, San Paolo.
Regional locations are particularly
good for high analytical workloads.
You can localize your compute and storage together
into a single region.
Now for multi-region, think of it
as a large geographical area, such as the United States.
You can use multi-region when you
want to serve content to data consumers that are outside
of a Google network and distributed
across large geographical areas.
Generally, you should think about storing your data
in a location that is convenient or contains a majority
of your users of your data.
So for instance, for EU data, you
might choose your bucket in EU.
And for US data you want to choose your buckets in US.
Now lastly, for dual region, is a specific pair
of regions chosen by yourself.
This could be an example, it could be Tokyo and Osaka.
So you should choose a dual region
when you want similar performance advantages as
regional, but also want higher availability that
comes from being georedundant.
KUMAR NACHIKETA: So sounds like multi-region and dual region
both are georedundant storage, right?
Can you help a bit more with specifics,
when to go for dual region over multi-region?
NISHANT KOHLI: Sure.
So multi-region is great for customers
who want to push data outside of the Google network.
Use case like content serving comes to my mind.
And when you're thinking about dual region,
think customers looking for true active active workloads,
but also looking for serving high analytical type workloads.
So those are the two scenarios think about when content
servings for multi-region and for high analytical workloads
and providing true active active namespace for dual region.
KUMAR NACHIKETA: We know our focus is dual region
enhancements today.
Help us understand what those enhancements are
and how can these further benefits benefit our customers.
NISHANT KOHLI: Sure.
So we'll be talking about really two major enhancements here.
Until now, we had fixed pair of dual region--
for example, Tokyo and Osaka.
Customers did not have the flexibility
to pick their own region.
Now going forward, our first enhancements
give that flexibility where customers
can choose the region pair of their choice
within the continent.
So this helps customer choosing two regions which
are relevant to their workloads and can take performance
benefits through localizing their compute together
with the storage.
So in this case, scenario might be Oregon and Virginia
as the two region of your choice in the US region.
The second enhancement really is around turbo replication.
Customers, especially in regulatory industry,
have tighter RPO requirements.
And RTO with dual region is always zero.
And for RPO for high demanding replication,
customers now have a choice to choose premium data
replication, which gives them a 15 minute SLA-backed RPO
guarantee.
Now with turbo replication's truly differentiating
capability ensuring 100% of your data
is replicated within 15 minutes or less.
This is globally unique.
And single namespace ability to select custom regions
and also opt in for turbo replication
can absolutely be handy for many of our enterprise customers.
Kumar, so now that I have explained the enhancements--
so typical scenarios-- maybe let's
deep dive into the dual region capabilities.
Can you walk through the architecture
and how can we achieve this functionality?
KUMAR NACHIKETA: Sure thing.
So let's look at the dual region architecture and some
of the secret sauce, which works behind the scenes.
Say we create a bucket and we choose two regions--
say Singapore and Taiwan in Asia, in this example.
So when you create the bucket, despite having
two regions in the back end, as an end user,
you have just one bucket.
And you get one global endpoint to interact.
So it truly provides universal namespace and, Nishant,
you have emphasized that a lot as well.
So you don't need to worry about managing the replication
at the back end.
We take care of that.
And you don't need to manage two buckets now.
So this is major, major important.
Now for end users, the data is strongly consistent.
And for all the clients, read after write
works just out of the box.
I think it's a pretty big deal for such massively distributed
storage system.
Let's see a few things behind the scene now.
The way Cloud Storage handles metadata and data is different.
So metadata is always synchronously replicated.
So 100% of the time, metadata is same across both regions.
Data, on the other hand, is asynchronously replicated.
And Nishant, you already mentioned
about our recent announcement of turbo replication.
So customers have two choices now--
either to stick with default replication, which
is designed to provide georedundancy
for 99.9% of newly written objects
within a target of one hour.
So newly written objects include uploads, rewrites, copies,
and composition.
And when you choose turbo replication,
that is SLA-backed with a guaranteed 15 minutes RPO.
That is just amazing.
Now let's understand a bit how [? read-write ?] operations
works here.
So first of all, for end users or applications,
[? read-writes ?] can be served from both the regions.
And that's why it's truly active active.
Another major advantage-- let's look at write operations.
So as you can imagine, for every new write,
metadata is synchronously replicated across both regions.
And data replication is based on the replication
mode you selected--
either default or turbo.
But it will be asynchronously replicated
with different target time.
For read operation, let's say the client
in the region Taiwan has requested to read an object.
So metadata requests will always go to the local region,
which in this case is Taiwan.
For the data, if it was written by a client in Taiwan,
it's already there.
And it'll be served by the local region--
and in this example, by Taiwan region.
But what happens if the data was written
by a client in Singapore and data is still to be replicated?
So since the data is not yet replicated to Taiwan,
the read for data request will be
redirected to Singapore region.
But I want to re-emphasize that it
is transparent to the client.
And eventually, the data will be replicated to Taiwan.
And it will be served from Taiwan for future requests.
Again, this request handling is all happening in the back end.
End user is making all the read write requests
to the global end point of the bucket.
So that was basically dual region architecture
and how it functions, still at the high level, I would say.
NISHANT KOHLI: Wow, that sounds amazing.
And I think it would be great if we
can get a quick demonstration of this capability
as well for us to understand it better.
KUMAR NACHIKETA: Absolutely.
Let's straight get into the demo now.
So this is a typical HA architecture
with a load balancer to send traffic to a healthy back end
instance in two regions.
We will be simulating a portion of this architecture
to showcase the functionality of dual region.
We'll just focus on the compute layer
and how it interacts with the dual region bucket.
We'll use Compute Engine VMs in Singapore and Taiwan region.
So I have already created one VM in each region.
So I'll now create a dual region bucket across these two regions
as well.
So let me first create a bucket in the console.
We'll go to Cloud Storage browser and click on Create.
Let's call it dual-region-example-asia.
And here's the location type.
And I'll select Dual-region for our demo.
This gives me choices to select continent and then
two regions from the Currently Supported list.
Let's spend some time browsing through different options.
Notice we have turbo replication feature option here as well.
So if I select Americas as a continent,
I could choose any of these pair of regions.
For our demo, I'll select Asia-Pacific as continent.
And I'll select Singapore and Taiwan
as a pair of dual region.
Next, we need to choose a storage class.
So for the purpose of the demo, I'll just
go ahead and choose standard.
So we'll leave everything as default. Let's hit Create.
And the bucket is now ready.
All right.
I'll now [INAUDIBLE] in both the instance
and perform some tests with dual region bucket.
One thing worth mentioning-- by pairing
the VMs to the region of the bucket,
we ensure that reading from the bucket
is fast and has no egress charges.
I'll be using a simple shell script here.
It utilizes curl, so it takes bucket name, object name,
and where to write as inputs, and then
use the GCloud CLI to get an authentication token.
The curl command has the S flag, which puts it in silent mode
and suppresses all the outputs.
So let me run this script in Singapore VM first.
So when we invoke this to send the object
content to standard out.
And as the object does not exist,
we see object not found message here.
So what I'm going to do is to invoke the command every half
second from the Watch utility and monitor for the output
as we make changes.
So next, let me write some text from Taiwan to a file hello.txt
and copy this file to the dual region bucket
by using GCloud alpha storage CLI command.
So the copy is complete now.
And as you can see, we can immediately read this file
from Singapore VM.
We are using the global endpoint.
However, the back end physical endpoint is different.
And one way to see this is by running the command
for storage.googleapis.com.
So you can see, they are not routing
through the same back end.
Next, let's make some changes to the object
and observe the behavior.
So I'm going to modify hello.txt by adding more line.
The old version, as you can see, is immediately
updated due to the consistency of dual region.
And final test for us today is to illustrate
the consistency and replication is bidirectional
and it's truly active active.
Let's try to run these tests.
And this time, we'll write from Singapore region
and we'll read from Taiwan region.
So I'm just going to write a new file-- hello.txt--
in this VM.
And then upload it to the bucket.
And immediately, the file is updated with the new version.
And we can now read this from Taiwan as well.
Amazing-- super fast.
And as you can see, the changes are immediately
reflected at both the end.
So you can imagine due to this property,
even during a regional outage, the other available region
will be able to serve the data as it always
have full view of metadata.
There will be minimum amount of data inaccessible
as the data replication across region is asynchronous.
Overall, you see how easy it is to configure the bucket.
And through one bucket and global endpoint,
it's extremely painless for the application
to work with object storage.
And any back end failure is managed transparently.
So to wrap things up, here are some key takeaways.
We have brought two enhancements--
dual region selection, where you can now have flexibility
to choose two region pairs of your choice.
Second enhancement-- turbo replication,
where you now have the choice to choose a premium data
replication feature, which gives 15 minutes
SLA-backed RPO guarantee.
Turbo replication is truly differentiating capability,
ensuring 100% of the data is replicated within 15 minutes.
Next, I demonstrated how easy it is to configure the dual region
bucket.
It gives you a single global endpoint.
And it's strongly consistent.
And it offers seamless outage handling.
It's true active active in nature.
You have automated failover failback.
So it reduces your effort to zero from storage perspective.
Thank you so much for your time.
And see you again.
[MUSIC PLAYING]