[MUSIC PLAYING]
CARTER MORGAN: Welcome to another episode
of "VM End to End," a show where we have a VM skeptic
and a VM enthusiast come together,
hash out all things cloud, computer, compute-related.
Brian, thank you so much for being here today.
I've got some questions for you.
BRIAN DORSEY: Hello, hello.
I'm excited today.
Databases and VMs, it's my happy place.
And because of that, I wanted to bring
in Gabe Weiss, who's one of our colleagues
and expert in databases.
Hi, Gabe.
GABE WEISS: Hey, everybody.
CARTER MORGAN: Yes, happy to get into it.
I guess, I'm just going to dive right in, if that's cool.
I want to know, when it comes to data on the Cloud
with compute and whatnot, why not
just let a managed service run it for me magically?
Why would I want to run the database
level at the compute level?
Why does that make sense?
GABE WEISS: I mean, why not, indeed, right?
The dev answer that we always give, it depends.
There is nothing wrong with running a database
on a VM in the Cloud.
Let's be clear right away.
Cloud SQL, which is a relational database managed offering,
it runs in a VM.
So whether you're running in your VM or our VM,
you're still running in a VM.
The difference is, and why you might want to run your own
really comes down to one of requirements.
If you have a legacy system, you've
got an application you've been running forever,
and it has a set of requirements,
then a managed database might not satisfy those.
One of the things with managed databases
is we manage it for you, which means we restrict
what you can do with it, right?
BRIAN DORSEY: OK, so let's be specific.
What are the kinds of things that
might push you into running it yourself on a VM?
GABE WEISS: Right.
So I bring up-- like, if you've got this legacy
system, if you've got an application that you've
been running for a couple of decades,
it builds up over time-- what you add to it, when you have
requirements, what you need.
So a real good concrete example of this
is, take SQL Server, one of the big things with SQL servers
are stored procedures.
You've got these methods that run on the database itself.
Stored procedures have certain requirements sometimes.
Like, maybe they require a super user.
If you're running on a managed database,
we don't give you a proper super user
because we have to protect the database for you.
We have an SLA, a Service Level Agreement,
where we guarantee that the database is going
to be up all the time for you.
You hear some people talk about the nines.
The nines is your percent of how much uptime we guarantee.
Cloud SQL, for example, our managed offering,
is 99.95% available--
3 and 1/2 nines, which means it's
like 20 minutes of downtime a month or something like that.
So with these extensions, or the stored procedures,
we have to guard that uptime, which means
we don't give you everything.
So if you've got a system that requires that,
a managed system might not work for you.
CARTER MORGAN: OK.
That actually makes a lot of sense.
And I appreciate you for bringing in that perspective.
So now, my question is, what about someone
that maybe doesn't have their database in the Cloud
yet but they're thinking about switching over?
What's the process for that?
GABE WEISS: Right.
So you're talking about-- we call it the green field, where
new application, a new company that's thinking,
like, where should I put my stuff?
Everything's moving to the Cloud.
Let's be clear.
This what we're seeing in the industry,
is everyone wants their data in the Cloud
for a whole host of reasons.
You want to run analytics on your data.
You've got a global situation where
you've got an application that needs global access.
You're going to be in the Cloud.
So I always say, if you haven't got something already,
start with the managed service.
You don't want to set up all of the boilerplate stuff
that you're going to have to do to run
your own database-- the hardware management, all the operating
system and security updates, managing the database health
itself, making sure that the machine is healthy.
With a managed service, all of that's handled for you.
You don't have to do any of that stuff.
So if you're starting fresh, absolutely,
start with a managed service.
BRIAN DORSEY: Well, you just set it right there.
I love it when you're starting a new project,
and you get to set everything up from scratch.
But a lot of us are dealing with a lot of software
that already exists.
What do you do in that scenario?
GABE WEISS: So if you've got one of those legacy systems,
like I said, like the 20-year application you're
working with, there's a few different ways you could do it.
If your systems are all really tightly integrated,
and you've got it running and a tight cluster of machines
or something, you can just do that, the lift and shift.
Just take the entire thing, throw it up in a VM.
And now, instead of running in your closet,
you're running in our closet.
But we handle all of the machine health for you.
So that's absolutely a valid strategy.
If you need to get out of where you're at and get to the Cloud
first, just put everything in a VM and get going.
You do want to split the data out, though, right?
So strategically speaking, if your requirements get
to the Cloud first, do that.
Lift and shift up, then start working
on splitting of your data.
There are cases, though, where that might not be the case.
If you've got an application that, for compliance
reasons or other-- you have to have the application local,
but your data has now grown beyond where it can handle,
and you need to just get the data out,
splitting the data out is totally fine too.
But there's some problems there, right?
So the internet-- the internet's a problem, right?
So if you are in a situation where
you want to keep your application
but you need to move the data, that's fine.
But you should be aware that you'll
have to handle that on the application side.
So great connection to your data center.
Maybe you've got a 30-millisecond ping time round
trip.
If you're running locally, your ping to your database
is less than a millisecond for a really well-tuned
relational database.
So all of a sudden, your time is 30.
But it's not even that kind because, let's say,
for example, you're only allowing SSL connections
to your database.
The TLS handshake to establish that connection
takes two round trips back and forth
to establish the handshake before you get to the data.
So now, it's not 30 milliseconds to get your answer.
It's 100 milliseconds to get your answer.
BRIAN DORSEY: And then a bunch of apps
make a bunch of queries in a row, and so that just--
every one of those extends it out.
So what should people do in that kind of-- what
do you do about that?
GABE WEISS: I mean, you can't fix the internet.
The internet, you're kind of stuck with.
But the tons of connections, the queries that you talked about,
some applications, because they're
used to this quick turnaround time,
they'll just spin up a connection, ask their question,
shut down the connection.
If you do that a couple hundred times, and your round trip
is now not so quick, that's going
to wreck your responsive UI.
So instead, the first thing that I
tell people is, the first thing you want to do
is what we call client-side connection pooling.
Most libraries that you connect with-- so for Python,
we recommend SQL Alchemy is a great library
to use for connecting to your relational databases.
There's objects that you use to create the connection,
and there's a connection object, and there's a pool object.
That pool is a connection pool that
will establish-- it hands you a connection to the database,
and it holds it open for whatever you specify.
That means if you're making these hundreds of queries
every second, that's great.
It'll establish one connection, run all your queries
until there's some idle time, then shut it down.
So that cuts down your round-trip time
because the round-trip time for that security
is only on that initial connection.
And then after that, you're good to go.
CARTER MORGAN: Right.
This is really interesting to me because I
like that you said earlier, if you're starting green field,
go managed.
Use the managed service.
That's the easy move.
And then I can see how it gets a lot more complicated depending
on your needs.
The one thing I think that shouldn't be complicated
is actually moving to the Cloud.
And so are there tools or anything
to help make that a smoother process?
GABE WEISS: Like you said, it shouldn't be.
And there are absolutely cases where it is, right?
There's totally cases where just moving the data is hard.
One of the biggest problems with moving
your data is-- the boilerplate way
to do this in a lot of cases is what
they call the export/import, which is go to your database,
you hit a button, and it dumps all of your data
to this giant file.
And then on the new instance, you hit Import,
and it sucks it all into the new one, and you're good to go.
That's the old-school classic way that you do this stuff.
But if you've got terabytes of data,
it's kind of tough to do that.
And while you're doing the import, well, now what?
Unless you shut everyone down and say, OK, no on
touch the database.
We're doing a migration.
Now, you've got data that continues to come in
while you're importing.
So the gap between when you've hit the export button
and when you do the import, there's
this data that you're missing that you
have to catch up still.
There's other ways you can do it.
You can set up replication between two servers, which
basically means the current database that you're
moving away from, you set up as the primary.
You set up a replica, and you say, OK,
now move all the data over.
And it'll do it.
It streams it across the pipe.
It's kind of a pain to set up.
There's a bunch of little knobs.
To make that easier, we've got a service
it's called Database Migration Service, DMS.
And that basically handles it.
It still uses the database native ways to do it.
So, for example, on Postgres, we use PG Logical
as the way we do migrations.
And it will handle all of the connections, all
the connectivity.
You define a source object, you source database in the UI.
And then it creates a Cloud SQL instance for you.
And then it moves it.
Now, keeping in mind, this is only for managed service.
So if you're moving to a managed service,
this is the tool to do it.
If you want to run your own in a VM, you're kind of on your own
because it's not managed for you.
So you have to do all of these pieces.
You can still do it.
There's still ways to set up external replicas
and external primary instances to do it.
But you're on your own.
So DMS is the easy way to move to a managed service.
BRIAN DORSEY: OK, that's super cool because I have definitely
lived the try to schedule a window where you can shut
everything down with all the people involved,
and then hope everything works right,
and you don't have to revert or whatever everything.
And that's hard.
So it's nice to know that there's some tools
to make that easier now.
CARTER MORGAN: So to summarize what we just learned today,
just to recap everything you came and taught,
I still want to hold up that managed databases can
and should be used in a lot of instances.
But there are other instances where it doesn't make sense
to use them, where there's some capability that you need
that's not provided yet, or maybe just
the way your system is set up.
But if you're trying to transfer over,
then there's Database Management System,
which makes it a lot smoother of a process.
Did I get that?
GABE WEISS: Exactly.
And it all comes down to your requirements, right?
Like you said, nothing wrong with running a database
a VM as long as you're comfortable managing it.
If you don't want to worry about that,
managed services are great.
BRIAN DORSEY: So if people want to dig in and kind of learn
more about this, do you have any recommendations?
GABE WEISS: Oh, yeah.
So the DMS, the Database Migration Service documentation
on our site is a great place to start.
There's a lot of good links there.
It talks a lot about the strategies that I talked about,
and it will lead you into getting
comfortable with the idea of moving your data around.
For other stuff, there's a ton of blog posts.
We've got a lot of blog material on our GCP blog
around v managed service versus running your own.
There's troubleshooting.
There's a couple of ones that are like troubleshooting.
I wrote one on connectivity.
Networking is really hard just in general.
So I have a good post on networking
between different services.
CARTER MORGAN: Well, yo, thank you so much.
Thank you, Gabe, for being here.
Brian, as always, thank you for being here.
And if you're watching at home, leave a comment.
Let us know anything new that you've learned or picked up.
We definitely check the comments.
Thank you so much.
That's the end of this one.
[MUSIC PLAYING]