HUNTER WILLIAMS: So a little bit of background on myself.
I, as Bill mentioned, was on the Google Cloud Platform team.
I don't come from the media and content side.
I come from the cloud technology side.
So when Kevin reached out to me earlier this year
and asked me to come and join the team as they spun off
this business, I was really excited to go
on to the buy side of cloud computing
and see how things are from the other side.
So I joined about two months ago.
I came from the Google Cloud Sales Team,
previously with RightScale.
Most my focus in that world had been
on media and entertainment, gaming as well.
So I knew a lot of the people in the industry,
but it was really exciting to get
to join an actual studio that was building a cloud product.
So I want to talk a little bit about how Atomic Fiction uses
Conductor and then ultimately how Conductor uses GCP.
And to start, I want to talk a little bit about the history
of Atomic Fiction.
So in 2009, Kevin Baillie and Ryan Tudhope
founded Atomic Fiction.
They were alumni from IMD, which is a Disney company
or was a Disney company.
And they were also at Lucas for quite some time.
When they founded the company, they
wanted to build a business around new technologies that
had just come out.
So they were using the cloud.
At the time they were using Zync Render,
as Todd talked about earlier.
But as the company grew, and now that it's
125 employees, two locations-- well, three locations,
I guess, if you count the LA office-- the needs
of the business changed.
And so they decided, as many different studios do,
to build their own render pipeline.
But they were going to build this render pipeline
in the cloud.
So enter Conductor.
They hired a guy from DreamWorks who
had built very large-scale render farms before,
and he came in and built out a product on GCP
that took advantage of that minute-by-minute billing,
the fast boot up times, the fast shutdown times.
They could just simply deploy whatever
they needed as far as a render farm was
concerned in under a minute.
So how does Atomic Fiction use Conductor?
We use it for rendering on demand.
So submit a job, artist-- as Todd mentioned,
artists are expensive.
You don't want them sitting there and waiting
for any period of time.
So once they submit their job, we
want to get that back to them as quickly as possible.
So especially with high-priority jobs,
we'll just default map a single frame to a single instance.
One of the great things that we love about Google
is the sheer size of it.
So if we want to deploy 1,000 instances,
we can do that in under 10 minutes.
We can then run those for maybe 1,000 frame sequence,
and then shut those down with a 10-minute minimum,
say 11 minutes later.
And then all of this plugs into the Shotgun's cloud
solution, which is a great product.
We love using it.
We're doing some deeper integrations
with Conductor and Shotgun as well
that will allow us to feed data back and forth.
We're currently pulling data out of Shotgun,
but we'd love to push some data back into it
around how much does a shot costs to render?
How many times have you rendered it,
and what is the total cost of that to date?
And so a little bit about how Conductor uses you GCP
and why we love it.
So we use Compute Engine for our render farm,
as I'm sure most of you know.
But the things that we really like about Compute Engine
are, first and foremost the permitability.
We were on Amazon through Zync when
they were on Amazon before.
And we saw a lot of waste.
An average frame for us takes 37 minutes to render using Katana,
so if we only run that instance for 37 minutes,
we waste the last 23 minutes of that hour.
With Google's platform rollout, we're
able to run it for just that 37 minutes,
shut it down, not worry about it anymore.
Next is cloud storage and persistent disk.
So we use cloud storage in their Nearline archival system
to back up used assets, to back up old assets,
to store pretty much anything that we upload.
And it's become, for Atomic, our single source of truth.
The minute we use persistent disk with a custom NFS set up,
also some glusters, as you guys have heard today,
to do our file serving.
It works really well.
We have yet to hit a ceiling with it.
I'm going to walk through a pretty detailed example of how
we've done this on our most recent project,
and you'll see that we spike it pretty high a couple of times.
So a little bit about Conductor.
So Conductor is a new product.
It's pre-release.
You can go online and sign up to join the private beta.
We're happy to have you.
You're going to get a phone call from me
right after that happens.
We designed it with the artists in mind.
We've gotten a lot of guidance from the Atomic Fiction team.
It's great to have a really tight feedback loop
with coming out of a studio.
The artists are really the ones that we want to cater to here.
So we've built tools to integrate with NUKE,
to integrate with Autodesk or with Maya, to integrate
with other products in the industry,
V-Ray is another great one.
And we just want to integrate it in such a simple way
that an artist can build their scene and click
render in the cloud.
Send it to Conductor.
And then we give those artists a really simple UI
so they can go in and see if jobs are pending,
if they're uploading, if they're completing,
maybe view some log files if they
want to do some of their own troubleshooting.
And then for the render wranglers and the admins
out there, we build tools that make
the troubleshooting process just that much easier.
So we have complex job management tools.
You can update priorities on the fly.
You can restart, kill, pause, check the status of jobs.
You can build automated workflows.
So as you start to build priorities and things
into, say, Shotgun software, you can make sure
when a job goes to the cloud that a high-priority one drops
right into a queue and starts going right away.
And then for the finance and operations folks,
this is one of the things that we're
going to talk about in the next couple of slides.
It's an interesting thing.
Cloud costs have always been considered really opaque.
And a lot of people have been concerned about runaway costs,
IT leakage, people launching instances and not
shutting them down, people storing stuff and not deleting
it, and really dealing with runaway costs.
So one of the things that we took really
seriously as we started doing this was to focus on how do we
want to present that data to the artist, to the administrators,
to the finance people?
Is it on a daily basis?
Is it real time?
What we landed on is that we use Google's BigQuery
service and Datastore.
So we have all sorts of bread crumbs
in our code to be able to track when
it's launched, what shop they were associated with,
what they were doing.
And we feed all of that data into a huge table that we then
can build dashboards around so you
can see this show by shot by artist,
and slice that data in a lot of different directions.
So I want to talk through a real-world example here,
because I think that's probably the best
way to talk about cloud costs as compared
with maybe a more traditional data center approach.
So the most recent project that we finished
was Bob Zemeckis's "The Walk." "The Walk"
is the story of Philippe Petit who walked between the Twin
Towers in the 1970s with no safety equipment,
for those of you who have not seen it or heard about it.
As you would imagine, this took quite a lot of visual effects
time, because the New York skyline looks a lot different
than it did in 1970.
So we had to recreate all of the background, the towers, all
of that for 37 minutes worth of visual effects for this movie.
So it did take quite a lot of render hours.
It took just over 500,000, in fact,
which is 87 years worth of rendering time
on a single node.
This is all done on 16 core instances for our rendering,
32s now that they're out.
And we are also doing a lot of our comping
now on smaller instances, taking advantage
of just the full gamut that Google has to offer.
So you look at our cycle here.
So we started off-- you know, the months
are somewhat arbitrary, but asset creation, poorly
optimized renders.
That's what that early ramp is.
Months three, four, and five are October, November, December.
So that's the holiday dip.
People are not working for half the month.
This year was especially.
Everybody had about two weeks off.
And then you see us ramp way back up into production
and peak out there with quite a lot of render hours
in what was March.
Then you can see it drops back down to almost 0.
And then this now, this new month, it
is all the way down to 0 for this project.
So zooming in on March alone, because that was really
our big spike month, over the whole course of the film,
we used 9.1 million core hours.
We're normalizing some this data into core hours,
because we used difference instance types and things
like that.
But in March alone, we used almost three million
core hours.
You can see this is on a day-by-day basis
the breakdown of our render process.
So sort of a normal two-week cycle.
Then you see a big spike as we started
getting into the final film reviews with Bob.
And then you see one big spike, which
was the night before our big film review,
where we had to render a whole bunch of shots and scenes
and do a whole bunch of comping in that particular case.
You see it drops back down.
That was the day of the meeting.
We got some notes, spikes back up.
And then it goes down as we deliver the final project.
Coming from the cloud space for a long time,
this is like the perfect use case for cloud.
You could own the infrastructure, but where do
you own for?
Do you own for the two big spikes
there in the middle and deal with some delay for the bigger
one?
Well, that would have meant, in our particular case,
that the shots wouldn't have been ready for their film
review at 9:00 AM.
So the cloud is an obvious choice.
So we had a maximum spike to over 1,000 nodes.
A couple interesting stats on that.
It took us about 12 minutes to launch all those nodes.
That was pretty impressive from my perspective,
when I was watching it that evening,
watching the number bounce up.
And during our common peaks, not just in this month
but in other months as well, we see around 500 nodes.
So for sake of evaluation as we go through some of the cost
comparisons here, we actually took it below 500 to 400 nodes
to do a cost comparison, because at 500, it just
didn't make any sense to buy that many if you're only going
to be working on one project.
So this is Conductor's cost curve.
So this is for Atomic's actual costs.
These are actual rack rate bills straight out of our system
that we sent to Atomic.
You can see that it mimics very similarly that curve that we
showed in the previous slide.
But you can also see that the monthly loaded cost
of supporting 400 nodes, for most of the months there,
it's significantly higher than what they actually
would have needed.
Now, this doesn't take into account
the fact that we were working on other projects
at the same time, and there is some overlap there.
But if we go back and we analyze what
we need, when we need to hit spikes at the same time,
when we have production deadlines that overlap,
we think that this is a pretty fair estimate
of how much we would have had to spend
on an actual infrastructure.
So here are some summary notes on this example.
We used 9.1 million core hours, spiked over 1,000 nodes.
With local infrastructure, if we had
tried to support some of those spikes
with some optimizations added, we
would have ended up at 30% average utilization.
That's a pretty gross number.
You don't want to ever see that in a data center,
but it's sort of a reality of the fairly cyclical work
that we all do.
We estimate, by subtracting those two previous graphs
together or from each other, we saved about $800,000
on this show alone.
That's a pretty big number.
When we look at the bids, that makes us much more competitive
than some of the other studios that are out there offering
similar-style visual effects.
A lot of this we have to thank Google for.
I mean, billing by the minute, that's amazing.
We obviously would not be able to achieve
any of these sorts of numbers with hourly billing at Amazon.
The hardware is delivered on demand.
If you compare this to a three-year buy cycle,
you have to buy hardware and you're not
going to be able to get updates on it.
Google updates their hardware all the time.
So you're constantly getting the latest and greatest,
better network performance, better servers.
As things start to go obsolete, they're already updating them.
And then the global footprint.
So we have offices in Montreal that
are doing rendering-- Montreal and Oakland.
So being able to have this happen
in the middle of the country where
we can do caching between the two with Avere,
and allow Google to be our single source of truth
that we pull any texture, color, dependency down from
to do work on is amazing.
And the fact that Google's network is so vast
allows that to happen really seamlessly.
So I want to wrap up with a little bit of talk
on where we see the future of the cloud going.
We see further democratization of these resources and easier
access.
We're seeing, through partnerships
that Google is forging, that we're
forging with some of the vendors in the industry,
that there's an appetite to do cloud-based licensing.
So licensing as a service, pay by the hour, pay by the minute,
pay by the second in some of our partners' cases.
That's something that wasn't there before.
So we want to help lower the barrier
to entry so that people can get on the cloud
and can set these farms up and do their work easier.
At least in our case, we already see cost savings.
We know through a lot of the discussions
that we've had that people have done a whole lot of analysis
on this, and they don't think the savings are there.
We believe that they are, and we're passing those
through to our customers, or Atomic Fiction
is in their bids.
Our bids are very cost competitive
and significantly lower than a lot of the competition
out there.
And it allows us to afford higher quality artists
so the end product ends up being much higher quality.
And then in the long run, we do see the entire workflow
moving to the cloud.
We've already got things like Shotgun that's cloud-based.
People are starting to rendering in the cloud.
But as technologies come out for better compression
and for things like that where we could
put artists' workstations in the cloud,
we could put the full render pipeline up there,
and be able to stream back color-accurate scenes.
All of a sudden you think about the industry
and how many people are moving to Canada and to other areas
due to the tax benefits, you'd be
able to create a studio on demand
and simply give all of these artists Chromebooks or MacBook
Airs with big high-def monitors that they can use,
and you end up saving a whole lot of money
on overhead by being able to do that.
So with that, I'm going to be around for next hour or so.
I know that I'm standing between you guys and a delicious Google
lunch.
So come ask questions.
I'm happy to chat.
You can find out more information about Conductor
by going to renderconductor.com and signing up for our beta.
Happy to get in touch and chat with you
about what we're doing.
And thank you very much.