My name's Jeff Kember and I do come from the visual effects
industry.
I was most recently a CD supervisor,
and I speak VFX and cloud.
So really happy to be here today.
I'm going to be giving a talk on Google Cloud Storage options
and just what it all is.
When I joined Google, I wasn't quite exactly
100% clear on all the different storage options
and how they worked and why you'd use them.
I'm also going to talk a little bit
today about getting your data to the cloud and how that looks.
So from a Google Cloud Storage options standpoint,
Google Cloud Storage is a giant block store.
So if you have onset data-- let's
say you've gone and shot a huge amount of imagery onset
and you want to put it somewhere,
the standard storage is a great place to put it.
It's super-low latency, it's almost limitless in size,
and it's $0.026 a gig so it's really inexpensive.
We have a less expensive option for $0.02, which is the Durable
Reduced Availability Storage.
So what does that mean to the VFX community?
That means that the latency is increased a tiny bit.
That's a really great place to park a significant amount
of data that you want to be able to do
read-and-write operations for.
If you want to be able to copy this data to a file system,
if you want to be able to copy it to local SSD
as on VMs, which we will talk about in a few minutes,
if you actually want to do transcoding or rendering
work loads against this, DRA is a great place to put the data.
We'll also talk a little later about Nearline storage, which
is our archiving solution, and go forward from there.
As far as storage on the actual machines go,
when you spin up a VM, you have a few different choices
on how you want to get the data available to the machine you
want to run it on.
So this slightly complicated diagram here
talks about persistent disk.
PD stands for persistent disk.
It's network storage, and it's available
both with enterprise-level spinning disk
as well as with SSD.
And we configure them in 10-terabyte sizes.
We also have local SSD attached to the machines,
and that is next generation.
It's super fast.
Think of it less as a SATA option
and more of a PCIE option.
It's almost 700,000 IOPS and it's 1.5 terabytes of capacity.
So that's crazy fast.
Instead of thinking of it as fast disk,
we think of it more as slow memory.
It's about the speed of DDR2 memory.
So if you want to deliver something like a ZFS file
system, for instance, you can spin up a VM,
throw a ZFS on it, attach 1.5 terabytes of storage to it,
and now you're serving up 1.6 to 1.7 terabytes of data at memory
speeds, at line rates.
That's about 8 gig per second on the file system.
You can also attach 10 terabytes of PD to that, as well.
So that's a really meaty little file system
that you can put up, which is open source,
and you're only paying for the VM.
If you have raw camera data and you
want to perform transcode operations, for instance,
on it, you can store it on PD.
You can also copy it over to local SSD.
So the use cases I've identified here
are transcoding, fluid simulations, Nuke, ZFS,
Read Cache.
So we have customers who want to do
fluid simulation on the cloud.
And from a storage standpoint, they'll
copy the alembic data and the scene data over,
they'll do the Naiad or Bifrost or RealFlow fluid sim on it.
But they'll use the super-high I/O capabilities
of that machine to be able to write all the cache data.
So the point is that with local SSD you can get really high ops
and you don't have to worry about using any network
bandwidth to do it.
I've had as much as 30% of my simulation time
be eaten up by copying files across the network
to an Isilon.
Not that the Isilon wasn't fast.
It was just, that's 20 to 40 gigs per frame
of data you're moving, and that does impact simulation times.
So from a network connection standpoint,
we've talked significantly about how many points of presence
Google has and how strong the Edge network is.
The bigger issue is, as visual effects companies and media
companies, how do you get your data onto the cloud.
So we have several different options.
We have customers who light up dark fiber
between their location and a peering location for us.
We have customers who will work with a partner such as Sohonet,
where they'll buy a 10-gig interconnect.
And they'll only pay for one gig to the outside world.
But they'll be able to get a much higher bandwidth
connection to Google and just pay a fractional increase,
not what the full load would be if you
wanted to have that go to the outside world.
Google also has a VPN offering, which
is a fully managed solution, which can
deliver gigs of connectivity.
The nice thing about that solution
is, you just tell us what you Juniper or Cisco box you
have on premises, or other.
And we have a profile that works for it.
It's managed.
It just works.
And if you want to run commercial or private queuing
software, that all works over the VPN.
It's just an extension to your network.
So as far as the transcoding workload goes,
so the green represents you're on premises or on set.
And this is a copy-to-Google cloud storage.
So you put in the punch ECS, and you
can park all the data there.
When you want to run a transcode workload on it,
you can spin up VMs using FFmpeg, just open source.
We can also work with PandaStream and Zencoder.
A local SSD is really the secret sauce for this.
Because of the signature of the job,
there's a huge amount of random reads and writes
that occur in a transcode.
So you can spin up a 8-, 16-, 32-core VM and run multiple
transcodes in parallel.
And then you can write all that data out to disk,
and the local SSD will handle that for you.
If you want to run more, we have customers
who are using a product like Deadline.
Or you can use pipeline effects, Cube or Tractor,
or your own queuing system.
The cool part about that is that the queuing system
will spin up the VMs, copy the data from GCS
over to the local SSD, unpause the job, run the transcode,
and then, after the job's finished,
copy all the data back to GCS, which is here,
and then spin the VMs down.
So you only pay for what you use.
You can go massively parallel with it.
We've had customers who have 5,000 hours of content
that they want to transcode.
In this example, it was all German.
And then they needed to be able to deliver it for distribution.
And they were used to having a 7- to 12-day delivery cycle.
And we said to them, well, how many hours you have?
And we got the file sizes and figured
what the longest transcode was.
We were able to deliver the job in just over a day.
That was the longest transcode.
So we just ran it all parallel, all at once.
And it was the same cost to them,
whether it would be just over a week or a day.
That was one of the nice things about that.
So you've parked it on DRA at this point.
So it's on Durable Reduced Availability Storage.
So it's the same system as GCS, but again, slightly higher
latency.
But you've already done your copy,
you've already done the transcode,
you've put it back to DRA.
At this point, you can push it out to the network.
You can use one of our partners like Fastly
if you need to use a CDN.
If you just want to deliver it to another provider
and they're going to handle the distribution, this is your out.
Now, you still want to, of course, retain this data.
So now you can put it on Nearline.
So the magic part with Nearline is, it's $0.01 per gig.
So again, on the far left-hand side,
we've gone from GCS at $0.026 to DRA at $0.02 a gig,
and again, as much reading and writing as you want.
And then, when you move it over to Nearline,
that becomes a $0.01 per gig place to store it,
and it can live there as long as you want it to.
If you wish to read that data back,
it's going to cost you $0.01 to do so.
So the nice thing about this is that the data
is available to you almost instantly.
The latency from first read to byte is two to five seconds.
So it's not hours, it's just seconds.
It comes back extremely quickly.
I'll have a little slide with some more details on that.
But we have customers who will put their data in Nearline
and they'll store it there.
If they wish to do operations on it, instead of being worried
about how many times I'm going to read it
and how many times do I get charged that one-cent fee,
they just flip the switch and move it back to DRA.
And it sits in DRA and they perform
all the reroute operations on it for weeks or months,
potentially.
I realize I can't give historical data,
as Nearline is still very new for us.
But those are the use cases that we are using, moving forward
with it.
So the compute engine we're running on
is really exciting for me in the fact
that it's crazy-high performance.
You are only paying for what you use.
The real differentiator for us is the local SSD performance
for the transcode, for the simulation workloads.
From a storage standpoint, this previous transcoding slide
that I had, it's the same work case
if you want to run Nuke jobs, for instance.
If you've already rendered the image in the cloud, it's there.
And you parse the Nuke file when you submit it
through your scheduler.
Only the elements that have changed
are going to be uploaded.
And you're able to benefit from the super-high-I/O output
of the local SSD drives that are physically attached to the VMs.
So we have customers who are using Nuke on this.
We have customers who are using FluidSIMs.
They have, for their proprietary,
Fire systems that run on top of our systems.
And again, all of them are using this really high-I/O storage.
And what it does for them is saves their on-premises file
system.
I'm sure if anyone has run large rendering systems, where
you have tens of thousands of cores
and you have a significant amount of compute happening,
if you load your file system up with a large amount of I/O
for a FluidSIM or any other type of simulation
or compositing workload with deep data, where you're looking
at an order of magnitude more data per element,
you're in a scenario where that file system
performance can degrade.
And then you start starving out your CPUs.
And as soon as that happens, you have
to start throttling one and favoring another and such.
If you move the high-I/O workloads to Google Cloud
Platform, you don't have to worry about your file system
performance on premises.
You can focus on having 100% of your machines
running renders or a smaller percentage
running comp and sim jobs, but you're not
negatively impacted by that.
So as far as Nearline storage as a slide goes,
this details a little bit more of the relevant information.
So it's $0.01 per gig to store it, which is super inexpensive.
It's one gig when you read it.
So again, if you want to read it more than once,
or if you really intend to be doing operations on it,
you want to put it on DRA at least,
or GCS, depending on your latency needs.
We have customers who are excited that, OK,
hear the $0.01 and they're like well, OK,
so everything's $0.01 now.
You're like, well, no, it's really
$0.02 if you want to do operations on it
and $0.01 if you want to park it somewhere.
There is a one-month minimum charge to put it there.
So it's priced in such a way that it is not intended
for production workloads.
But it's priced in such a way that it's
intended to be used for ultra-long-term storage.
Another nice aspect of being able to use Nearline
as an archiving tool versus tape is,
you're going to make two to three copies on tape,
and then you have to transport them and store them,
and you had to pay for the robot and someone to run it,
and then you have to track and manage and deal
with all of this.
The nice thing is you can modify the size
of the footprint of the data that you've put on Nearline.
So if you take a 400-terabyte show
and you need to put it somewhere,
then you can throw it onto Nearline.
And then you can put a pipe technical assistant on it
to go through and look a shotgun and figure out what can stay
and what doesn't have to.
And you can reduce the size of the show.
I've had shows that were 400-plus terabytes
and we've reduced them down to 225 terabytes on disk,
because of course we didn't backup the temp
files, obviously.
But you go through and you figure out
you don't need 17 versions of Character X.
And you don't need a whole bunch of temporary MARI archives
that were 250 gigs apiece.
You can go through and say, we only
use these two and these were live
and these are the latest versions.
So the ability to change the footprint of the archive
really affects the long-term storage cost as well as
the organization of the project.
So you can see it's super-high durability.
The retrievals are fast.
They're only a couple seconds to get the data back.
I'm sure everyone here has done restores or made
restore requests.
And you've waited anywhere from a couple of hours
to go and get that tape and load it in the robot
and restore a 20-gig character archive.
And then you realize, oh, well, that texture was actually
borrowed from over here so you have to do a second pass to get
the data back.
Or you restored a significantly larger portion
than what you needed because you wanted
to make sure you got it all.
The nice thing with this solution
is you can traipse through the file structure,
grab the files you need, and just bring
them online right now.
And the fact that the more you store,
the higher the throughput is.
So those customers with large pipes,
if they have hundreds of terabytes or petabytes of data
store, they have a significant throughput
and it's predictable.
And they know what the cost is going to be,
and our APIs are known.
From a pricing standpoint, both the compute and the storage
have been heading down with Moore's Law.
So you can see our curve from 2006 to 2015.
It's continuing to go in that direction.
So customers who've been using our Ivybridge technology,
we've rolled out Haswell in a number of our zones,
and we're continuing to update and refresh our hardware.
In that context, you're able to get the increased performance
at not the same cost but at a lower cost,
as we continue to reduce our costs,
as well as increase our storage capacities and availability.
So from just an overall use case standpoint, with Google Cloud
Storage most customers here are going
to be putting their data on GCS from an upload standpoint
and parking it there.
And then they're going to be spinning up
VMs as file systems.
And they're going to use ZFS, or they're going to use Avere,
or they going to use Gluster.
Avere is here to talk today.
They're a fantastic solution that most everyone in the room
knows about.
They have it on prem.
Our cloud offering is excellent.
You can have between 3 and 30 VMs
and it can scale larger than that.
You get to use our fast, low-latency network fabric,
and you're able to use our persistent disk SSD based
for the back-end storage.
So we are able to serve up file systems that
are capable of having tens of thousands or hundreds
of thousands of cores point at them simultaneously
and be able to deliver that throughput that you need
at a really high read rate.
It's also very nice to be able to have something like Avere,
in the fact that you can use it as simply a caching appliance.
So you don't have to pre-populate it
if you don't want to.
You can spin it up and read through it.
And then the data is cached.
Your hit rates are high.
For smaller jobs that are potentially
hundreds of machines, you can point at a ZFS file system.
And Bjorn Leffler, the engineer who did our ZFS file system
is here today to answer any questions
you may have about that.
He's in the blue shirt at the back of the room waving hello.
It's exciting in the fact that it
can stand up to hundreds of machines hitting it.
It'll deliver eight gig per second all day long.
And as I mentioned before, it's backed by our local SSD.
You can also have multiple ZFS file systems.
You have your texture on one, and you
can have your alembic data that churns
a little bit on the other, and you can point
multiple mount points at it.
So feel free to come and talk with me
after, or Bjorn, about some of the different VFX
workloads for storage.
Happy also to answer questions you
might have about getting the data to the cloud.
Just to wrap up here, putting the data on GCS,
spinning up a file system to be able to run
compute loads against it.
Once you've completed your compute loads,
if you were on a file system, you
can put it back on GCS to park it,
or you can copy the local SSD data back to GCS.
And then you can spin your file system down
if you don't have production workloads to run.
And finally, you can use our network to deliver the data.
And then, once the show's wrapped,
you can take the data as you've been finaling the shots,
you can move it off to Nearline, and the entire show
can sit on Nearline for as long as you want it to.
Yet you can bring any assets back online in seconds,
to be able to get at.
And I think everyone can agree that that's a really useful use
case for storage, being able to get at a show from six
or eight years ago and be able to pull those assets online.