RON PONTAFARO: Hi, everyone.
It's me again, Ron from the Product Team
for Databases here in Google Cloud.
Today, we're happy to host Sergio Salvatore, a Senior
Director of Engineering from Vimeo,
and he's going to tell us about how our databases help keep
their platform up and running.
From the Google side, our host is Pranav Nambiar,
a Director of Product Management here for Google Cloud
Databases.
So let's start.
Sergio, can you introduce us to Vimeo please?
SERGIO SALVATORE: Absolutely.
Thank you for having me.
Vimeo is a leading player in the growing video SaaS market.
We provide a complete suite of video tools for every size
and kind of business from local shops to global enterprises.
We're serving over 200 million users across more than 190
countries, and rolling out new capabilities
that we believe will unlock end-to-end adoption of video
across departments and teams.
We recently launched a new enterprise product,
a corporate video library that will change the ways
organizations create, share and organize their video
across teams and departments.
PRANAV NAMBIAR: Nice.
Hi, Sergio.
As an all-in-one video solution, Vimeo
is clearly in a space where there's
a very high bar for customer experience,
and there are a lot of challenges
as well along with it.
Can you share with us some insight into the key challenges
in your industry?
SERGIO SALVATORE: Yeah, absolutely.
So I think one of the big challenges
for me is the long tail of content that we have.
So certainly, we have some content
that's very popular and very easy to cash, in that sense,
that it's sort of predictable.
We can put things in fast buckets
and make them available to users relatively easily.
But we have a very long tail of content
that we want to provide the same user experience for,
and doing that sort of intelligently and efficiently
is a challenge just in general.
And as I said before, we're talking
about 200 million users.
About 1.6 million of those are paid subscribers.
There's 350,000 new videos added per day
and 100 billion views that are happening,
so that's just tricky.
PRANAV NAMBIAR: Great.
Can you also tell us a little bit more
about your use cases and the key managed database services
that you use?
SERGIO SALVATORE: Sure.
I can certainly give you a sampling.
We use Bigtable for video transcripts
and we use Cloud SQL for some sort of lower-volume satellite
services, but at the core of things
that we need to have highly available,
we use Cloud Spanner, and in particular, we
use it for a video metadata indexing service.
So as you might imagine with a company like Vimeo,
people are uploading content to us all the time.
We store that in Cloud Storage, and then we
transcode it in various qualities
to optimize for various devices.
And we need to keep an index, a catalog of all those files
that are being created so that when we need to play it back,
we can find where all of that stuff is
and play it back for the user.
And we need to have a database that's very scalable and highly
available.
If that index is down or unavailable or slow,
then playback can be compromised,
and that's not something we want to do.
And not to mention that, as the business grows,
we're just adding more and more content to this database,
and we want to make sure that we can scale easily there.
And we use a multi-regional spanner instance,
so we get 99.999% of reliability and availability.
We're using about 16 spanner nodes
right now that cover 15.8 or so billion rows and about 4
and 1/2 terabytes of space.
PRANAV NAMBIAR: Great.
Well, awesome to hear that you're
using our transformative databases such as Bigtable
and Spanner.
Definitely, at the scale at which
you're operating, the number of technical challenges
that you kind of need to conquer every day, literally,
I'm sure many of our other customers
would also like to learn about the insights
that you have had during this journey.
So can you maybe provide us a little more details
on what made you choose GCP and these specific managed
databases, and your journey to GCP?
SERGIO SALVATORE: Absolutely.
So we initially engaged with Google
because of the excellent performance
that there was between GCE and GCS.
I think that was something that was
going to allow us to create a really
high-quality playback and upload environment for our customers.
So the first version of things was basically
building, packaging and delivery services
in GCP with files backed by GCS, and that was successful for us.
And it sort of set the stage for us
to be able to migrate out of what
we were in at the time, which was a co-located data
center, which many companies of our age were in.
And we were basically running most of our services on prem,
and they were backed by MySQL, and we
basically took all of those services
and lifted them out of the data center and moved them into GCP.
And initially, that worked great for us,
but we realized at that point that we
felt like we were outgrowing MySQL,
in particular for the metadata indexing
service that I was talking about before.
So we started building and designing
a next-generation service that we could use Spanner
as the backing store for to get those kinds of availability
and scalability requirements that we wanted.
And the advantages of that being a managed database basically
is that we don't have to worry about.
We don't have to worry about patching and upgrading
and scaling.
Scaling is very simple.
I can just press a button and scale up to more nodes
that I need, and we had some internal expertise
with Spanner, which was very helpful.
So it made it sort of a natural choice for us
to use for this case, and I think it will definitely
allow us to scale to the heights that we want to in the future.
PRANAV NAMBIAR: Nice.
In today's world, frankly, when every company
is kind of looking at moving to the cloud,
one of the biggest hindrances or challenges is migrations,
and you folks have already, you know, gone through that journey
and actually completed migrations very successfully.
So as part of this, could you maybe
share with us some of the high-level insights and lessons
learned from your migration experience
that you think other customers may find really helpful?
SERGIO SALVATORE: Yes, definitely.
And I've done this a few times in my career.
And I think one of the main things I can suggest
is trying not to change your application too much before you
move it into the cloud.
Try and keep the number of variables minimized
so that when you lift your application out and start
deploying it in the Cloud, maybe the issue that you're
going to have is with your cloud configuration,
and it's not the application itself,
the software that's somehow doing something different
because you just changed it.
So I think that would minimize variables and minimize
the types of things that you might have to debug
when doing things like that.
PRANAV NAMBIAR: Awesome.
Could you also provide some details
on your overall architecture?
SERGIO SALVATORE: Absolutely.
Yeah, we run most of our applications today on GKE,
and that's quite an evolution, I think,
from where we were before.
We were running on prem in the data center with bare metal,
and now we're running in a containerized environment.
And it took us a bit to get there.
We sort of undertook containerizing the application
as part of building a new system to run asynchronous jobs.
So we wanted to do that in a different way,
and we wanted to leverage GKE and preemptive instances
in particular to be able to take advantage of lower costs there.
And obviously, one of the first things we needed to do
was containerize the application,
which was not really an easy task, but we got it done.
And then we built that system to manage asynchronous jobs,
deployed that out into production,
gained some experience there with how
things were working in GKE, and then started
to feel confident with what was happening there,
and then took those learnings and migrated
the rest of the application to run on that same stack.
And we're continuing to do that.
Most of the environment now in GKE, though.
And we're using other things sort of as connective tissue
for talking about Pub/Sub for asynchronous communication.
And then as I said before, like Spanner and Dataflow
and BigQuery, to be able to do things
like business intelligence analysis and optimization
and things like that.
PRANAV NAMBIAR: As an advanced user of databases,
I'm sure you have used pretty much every access pattern
and functionality that we have on offer.
When you look at all the features that you have,
let's say from Cloud Spanner, what is the one thing that you
feel is really useful, and you would like to share it
with the rest of our customers?
SERGIO SALVATORE: Yeah.
I would say the point-in-time recovery functionality
has been really useful for us.
We've been migrating data from that MySQL table
into this new system backed by Spanner.
And as you might imagine, when you're
doing that kind of migration, there's
some transformation that happens in the process
and some bugs can crop up in there.
So when we're doing some test migrations,
we're able to just run our process,
migrate things into Spanner, then run
some analysis on that migration process
and have a sense for whether it worked or not,
and then shake out any bugs that might crop up there, and then
basically click a button and go back to where we were before,
and run the same process again and see
if we've improved things.
So that's a pretty powerful thing to be able to do.
If we get corrupt data, we can just go back
like we have a time machine.
And then sort of along those same lines
is being able to read from stale time stamps,
is kind of a cool thing.
Sometimes, it's useful to know what
the value of a row or a column was,
and to be able to go back in time there and discover
that quickly is really cool.
PRANAV NAMBIAR: Great.
Great to know that point-in-time restore and stale reads
has been very useful for you.
Let me switch the question around.
What if you had a magic wand and you
could make a wish for a specific feature on Cloud Spanner?
What would that be?
SERGIO SALVATORE: Yeah, so I think
when we were starting our Spanner journey.
We had to think pretty deliberately about the topology
that we were going to choose, and that
wasn't a difficult decision, because we
knew we wanted to use multi-regional instances
eventually.
But at the time that we were designing the system,
we didn't have an initial need for that,
and obviously, there's an expense associated with that.
So it would have been nice to be able to start with a smaller
deployment that maybe wasn't multi-regional
and then be able to click a button later and change
it to multi-regional after the fact.
We probably would have made use of that
so that we could save some costs there and not necessarily have
to reserve a whole bunch of resources
that we didn't need at the time.
PRANAV NAMBIAR: Got it.
Well, thanks a lot for your feedback.
I think the good news is, this is something
that we're actively working upon,
and so you should be able to hear more about this shortly.
Thank you so much, Sergio.
SERGIO SALVATORE: Thank you.
RON PONTAFARO: Thank you, Sergio,
for joining us today and telling us about your database story.
Thank you, Pranav, for hosting.
From me and everyone here in Google Cloud Databases,
thank you for joining us and see you next time.