DAVID NETTLETON: Hi, everybody.
My name is Dave Nettleton.
I'm a Product Manager here at Google Cloud.
And I'm looking forward to talking to you about where
to store your stuff.
So I've been with Google now for about five years, working
on a range of different storage products throughout that time.
I'm just excited to share with you
about some of what we've built over that time period
and where we are.
So agenda-- we'll cover three main topics.
So first of all, I'll just sort of step back a little bit
and talk about how the storage landscape has evolved
over the years, actually starting with from on-premises
through to the evolution in Cloud.
I'll talk about storage, and I'll also briefly touch
on the evolution of databases as well
and some of the other options where
you might store your data.
I'll dive in a little bit in some of the storage
options for Google Cloud.
And then end with a wrap-up on where to learn
and some more follow-up information.
First of all, just a safe harbor statement in case
I make any forward-looking statements.
And with that, let's dive in.
So a brief history of storage and databases.
So let's start with the evolution of storage.
And different categories of storage
have developed over the years.
These are the main ones.
And maybe starting at the left is the easiest one
to think about, which is direct attached storage.
So this is storage that is physically
attached to the local machine.
So your laptop has this, a desktop machine
would have this, where the storage
is physically attached to the device,
the file system is sitting locally,
and the applications are running on the device as well.
So these are super high-performance devices.
They're great for day-to-day work.
One of the downsides, of course, is
it's a very sort of localized and sealed system.
And you can't share, easily, data with everybody else
from the device.
You have to go over the network.
So on the right there, you'll see three different ways
that data is often shared over a network.
In the first one, a type of storage
called block storage is stored over the network.
So this is very popular for, for example, database applications
where the database server might be
running in a particular machine but the storage itself
is accessed and provisioned over a network
so that it can be managed and scaled separately.
And the file system is local to, for example, where
the database might be running.
And then the file system communicates over the network
with in what's called block sizes
or blocks of data that might come
in 8K, 16K, and 64K chunks.
And this is a very efficient way to send data over the network.
It's not always that great for sharing the data
because the way the data is managed
is very tied to the file system and that's
tied on that machine shared with the application.
Another very, very popular form of storage
which I think many people will have heard of
is file shares, file storage.
So here you can imagine there's a remote file
server for your organization.
And the application or the device remotely
accesses the data over that file share.
So many users can access that file system over the network
and then access that data to their application.
It's a lot more complicated to build,
because now you have to deal with many users
accessing the same data and you have
to build various locking and other semantics
to be able to manage that.
But it's a very, very popular, very powerful form of storage.
And then, finally, a class of storage
that was starting to emerge on premises
but really took off with the advent of the public cloud
was object storage.
So this was initially started as a type of storage
where you have really large amounts of unstructured storage
and you can reference each piece of it individually.
It was great for things like backup and archiving,
where systems could offload data to this cheaper form of storage
over the network.
Instead of a file system API, it had
an object API that made it super easy to scale very broadly.
And if we map these categories to Google Cloud for block
storage, either direct attached like the storage area network
or equivalent products, or local SSD for a direct attach,
and then persistent disk.
And these let you create an attached block storage to VMs
in particular.
Network attached storage, this is file storage, file share,
so Cloud Filestore is the product
that we have in that category.
We acquired a company called Elastifile to support us
in that a few years ago.
And we also have, actually, a number of really great partners
in this category too.
This is a category that's very rich, has
a very rich history and a very rich set of features.
We want to support our customers and partners with a rich set
of options in Google Cloud.
So we offer NetApp, Dell EMC, and DDN as options for that.
And then for object storage, our native cloud product
is Google Cloud Storage.
So a rich set of capabilities here
to support these categories.
So let me now move on to talk a little bit about the evolution
of databases.
And really some 30 or 40, 50 years ago,
this was really dominated by the traditional relational database
management systems, where customers
are looking for systems to capture transactional records.
Classically, the example is the financial record
where you're debiting one account
and crediting in another.
And you want to make sure that happens as a single operation.
The RDBMS systems became the mainstay
of very many enterprise applications
for managing their data.
Data was then lifted from these and moved over
to a data warehouse via ETL processes, Extract, Transform,
Load, that would restructure the data
and then provide it for analytics and reporting.
And this was then the data warehouse industry,
the database industry.
This is a "many, many billion dollar" industry
that grew very strongly for many years.
One of the challenges with this, though,
was that all these systems were what we call scale-up system.
So if you want more data, you'd have to attach more storage,
attach a bigger single machine to that.
And as data and the world became more connected,
more types of data were generated
that these systems couldn't really manage at scale.
So a number of different types of storage then emerged.
One of the earliest ones of these
was the data lake and Hadoop, where huge amounts of data
is getting generated.
Lots of unstructured data.
Customers weren't always clear what
they were going to do with that data initially.
So they want to store it at scale
and then process it and shape it for the application
or the questions that they had.
And then, beyond the large amounts of unstructured data,
there was types of data, sort of semi-structured data
that customers were gathering, either
in terms of sort of key values, where
lots of rows of data but also very many columns
but very sparsely populated, or semi-structured data like JSON
data and hierarchical data.
And customers needed to store this data
at really, really large scale.
And the traditional systems weren't always
super well set up for that.
So a whole new class of NoSQL systems emerged.
And then even more recently, customers
have looked to actually have some of the guarantees
and the features of relational database systems,
but now at global scale this has been one of the trade-offs
that the industry has sort of wrestled with,
is having consistent data, having it very highly available
and also being able to partition it and separate it out
for scale.
So there's something called the CAP
theorem that looks at the trade-offs between these.
And a lot of systems really focused
on the transaction consistency and the availability
and so weren't able to scale.
Now, something like Spanner is actually
a very interesting scalable relational database system
that lets you have and manage transactional records
at global scale.
And it makes different trade-offs in terms of it
provides consistency, it provides partitioning
and scale, and then uses atomic clocks to really manage
and very smart about how it deals with availability.
And if we map these to the products
that we have in GCP for the traditional database system,
we have--
Cloud SQL manages the database on your behalf offering
managed MySQL, Postgres, and Microsoft SQL Server.
We also have a bare metal offering for Oracle systems.
BigQuery is our really large-scale data warehouse,
very, very powerful, amazing system
for operating the really large-scale analytics.
For Hadoop and data lakes, Cloud Dataproc is our managed Hadoop.
And that, coupled with Cloud Storage
that I talked about earlier, offers really
large-scale compute and storage disaggregated between the two.
Cloud Bigtable and Firestore are our NoSQL products,
and then Cloud Spanner, a really unique, scalable RDMS system.
OK, let me dive in a little bit on a couple of the storage
options, some of their capabilities
and what you might use them for.
So Cloud Storage, so this is our object storage product.
So I'll talk a little bit about some of the use cases here.
So content storage and delivery, storing and serving
really large amounts of content and delivering it,
and delivering those files to users and consumers
over the internet.
A company like Spotify uses Google Cloud Storage
heavily for that.
Big data and analytics ML.
So the Cloud is great for being able to store
large amounts of data at really low cost,
and then only bring the compute to bear when you need.
So this disaggregation of compute and storage
is a super, super powerful way to store and manage and scale
your data.
Companies like Twitter use Google for use cases
like that, operating at really, really large scale
over both data and the compute and analytics side.
And then, as I mentioned earlier,
backup archival DR. This was the use case that really founded
object storage on premises.
But in the Cloud, it's actually a much smaller portion
of the use cases than many of the others.
But object storage still a great place
for storing large amounts of data for backup,
for archival, cetera.
I'll talk about a couple of the great features
that we have in GCS that support these use cases.
Probably the first one I really want to talk about
is having a single API that works across all of that data.
So we offer different classes of storage,
from standard through nearline, coldline, and archive.
And they offer different price points
for the amount of storage for the price
of storage stored per gigabyte and then the amount
it costs to access it.
So we offer these different tiers of storage.
We uniquely-- our archive and coldest tiers,
it's instantly available.
So that API that we use for our standard is
available across all tiers.
Same fast access to all data, millisecond access
to all of your data.
So all of that archival data is actually online and available
to you immediately to use whenever you need it.
And that's a very unique and powerful capability.
I talked a little bit about some of the big data and analytics
works use cases.
We store our metadata in Spanner,
so all our metadata is strongly consistent.
This is very powerful at the regional level for customers.
But actually one of the really nice features of Google Cloud
Storage is that, for our dual regional products, which
is where data is stored in two regions,
we can create a single bucket that spans those two regions.
And customers don't have to deal with which bucket
am I writing to, how do I deal with failover, what am I
reading from.
We provide a single namespace that
can access data in both of those regions, which
is super powerful for dealing with applications that
span multiple regions or that you
want to provide business continuity across those.
And a big highlight of some of our announcements at Next
are we're going to expand the capabilities in that
to allow customers to choose custom dual regions, so instead
of just fixed pairs that we'd offered previously,
offer flexibility in the choice of those regions,
and then also give an SLA around replication,
so a three-nines SLA for a 15-minute replication.
Persistent Disk.
So this is our block storage product.
Great product for attaching disks to VMs,
for example, is the classic use case.
If you're in a database in a VM, big data application, then you
might want to attach disks to that
so that you can get great performance
for those applications.
As well as running those enterprise applications,
you also want to be able to protect them
with high availability or backup.
So we have a rich set of capabilities around that.
Key features for this product lineup,
so actually a rich set of tiers all over,
giving trade-offs from lowest price performance all the way
through to extreme price performance with a product
we announced recently, called Extreme Tier,
giving provisioned performance up to 120,000 IOPS.
But we also want to make it really easy
to use this product.
So we make it easy to take those disks
and attach them to different VMs.
You don't need to stripe or prewarm for performance.
We really want customers to be able to very easily use
this product.
We also published a number of durability guarantees
around the product recently.
Another unique feature that I'd like to talk about here
is also regional PD.
So this is the ability to create a disk that spans two zones.
And then we synchronously replicate the data
between those two zones.
So this is, again, a super important feature
for enterprises that want to have
the very best business continuity
and high availability controls.
Filestore.
So this is the file share product
where customers are moving from on premises
to Google Cloud these might be video files that they're
sharing.
It might be web content, home directories.
A lot of very classic traditional enterprise
use cases here.
We've got a range of offerings in GCP.
Filestore's the product that we have.
Within that, we have standard premium tiers.
These were zonal products.
And now we've just recently announced
a new regional enterprise tier which
is based on the Elastifile technology
that we acquired a few years ago.
And this really lets customers, again,
build those really highly available, resilient
applications in Google Cloud.
And we've made sure, also, this is
easy to integrate with the rest of your applications,
integrating with all of the VPC service controls and network
capabilities that you need for a sort of control and security
boundaries, together with easy integration
with the main compute engines like Google Compute
Engine and Google Kubernetes Engine,
making it really easy through, for example, CSI drivers.
Another category of products I just
want to talk about briefly, Transfer Appliance and Service.
So I've talked a lot about the main object
file and block storage products that we have in the cloud.
But customers want to be able to easily bring data to the cloud
and move it around within the cloud.
And so we have a set of capabilities
to help customers with that.
So these might be large one-off migrations of data from a data
center shutdown, or it might be IoT use cases where
data is being gathered at the edge
and then we want to bring the data to cloud.
And so we have two main offerings here, Transfer
Appliance and Transfer Service.
Transfer Appliance offers the ability for us
to ship you a server, load it up with data, send it back to us,
and then we'll load the data into the cloud.
We're just recently announcing in online mode for that,
where you can flick a switch on the appliance that
says, hey, any data that goes here,
I'd like it to be immediately connected up to the cloud.
So if you can connect that appliance
to your network on premises, you can drop the data
onto the appliance, and we'll just
take care of moving the data up to the Cloud from there.
If you want more control over transfers,
we also offer Transfer Service, which
lets you deploy an agent on premises
to manage those transfers.
A number of great feature enhancements for that recently,
providing an API so you can have more programmatic access
over it, agent pools to give you more control and parallelism
over uploads and control over shared resources
like, for example, the network.
And then, finally, I really want to talk about some
of our partners.
Really believe strongly in having a strong partner
ecosystem with GCP.
Our customers want choice.
We want to support them in that.
For file, we've got some rich partnerships
with NetApp, Dell EMC, and DDN.
NetApp offering, for example, SMB capabilities.
Dell EMC, very high performance and scale.
And DDN for Lustre.
For backup and disaster recovery,
again, a rich set of capabilities and options
for our customers to choose from.
A couple of recent highlights, we've integrated with Veeam,
with GCS, so that you can now back up directly to GCS
as a target in the cloud.
Commvault offering a rich set of capabilities
around backup and data protection for GCVE,
our VMware service for GCP.
So finally, I'll just wrap up and leave you
with a few resources to help you learn more.
So first of all, just a shout-out
to some of the other sessions here
at Next, some sessions that we have in conjunction
with customers such as Nura and Sabre
about how they're using Google Cloud Storage
options to support their use cases
and then a deeper session on how to build resilient and highly
available applications on GCP.
I put some links in there to resources on our web pages.
And then, finally, a call-out to some of our training
and certification resources.
In particular the Professional Cloud Architect
as one of the highest paying industry
certifications of 2020.
Great way to invest in developing your skills for GCP
and hopefully also your career.
And so with that, I'd just like to say thank you
very much for joining me today.
And I hope you enjoy the rest of your day.