RICHARD SEROTER: All right.
Hi, everybody.
I'm Richard with Google Cloud.
Tammy is here with us today to chat about multicloud.
Tammy, tell us what you do again.
TAMMY BRYANT: Hi, everyone.
I'm an SRE at Gremlin.
RICHARD SEROTER: Awesome.
What does that entail?
What's site reliability engineering, for those of us
who don't always remember it?
TAMMY BRYANT: Yeah, a lot of different types of work--
incident management, being on call,
automation, reliability, making sure
that you are reducing downtime and keeping systems
up and running for customers.
And also, I specialize in chaos engineering.
I've been doing that since 2009.
RICHARD SEROTER: Awesome.
So what do you think of when someone
says the word multicloud?
TAMMY BRYANT: In the early days of the cloud,
I thought of on-prem plus a cloud provider
for compute, because that's what I first started doing back in,
I think it was 2011 when I was working at the National
Australia Bank.
And then a little time went by, and I
started to think of using multiple cloud
providers at the same time for both cost efficiency and then
reliability requirements-- being able to do failover,
for example.
But nowadays, I think of not just compute on the cloud,
but also new services that are available across
different cloud providers.
For example, we use Google's BigQuery and also Data Studio
for data, and then we use other cloud providers for compute
and other different types of services.
RICHARD SEROTER: So do you think that's one
of the primary reasons people do multicloud, is sometimes
accessing those unique services?
Is that what you see?
TAMMY BRYANT: Yeah.
I think it's actually a really amazing new way
that you can use the cloud.
Because when you're thinking of adding a new type of service--
for example, I was just doing some work
on identifying different types of services
that we could use for speech to text--
it's really good if you could take
each of the cloud providers' solutions for a test drive
and figure out what fits your needs best first.
And then also, you might consider building it
from scratch yourself.
But that could take a lot of time
and maybe it's not your core business.
So nowadays, to me, it's easier and faster to experiment
and then also plumb different services across cloud providers
together, which kind of reminds me of that '90s DOS game Pipe
Dream.
You're just able to plug and play, which I really love.
RICHARD SEROTER: That's awesome.
So what's the hard part?
So that can sound great, but where
are the things, at least in even setting that up,
where you'd be like, hey, be careful.
There's some sharp edges.
TAMMY BRYANT: Yeah.
There's a few different areas where
I've seen people get caught.
One of the most important things to think about first
is right-sizing your cloud use and also
keeping an eye on cost, performance, reliability.
I think a lot of folks maybe sometimes might assume that out
of the box, you just get all of these things
and it just works perfectly once you move to the cloud.
But there's a lot of mistakes that folks
can make, in terms of configuration being not set up
correctly, or just actually having limits in place where
they don't realize until they get a ton of traffic
and then suddenly there's an outage because they didn't plan
for that or have the capacity available to be
able to scale up.
So yeah, that's why I'm a big fan of chaos engineering,
even as we've moved to the cloud instead of being on-prem.
I started doing chaos engineering
on-prem for large-scale disaster recovery and failover.
But now with the cloud, chaos engineering's
a great way to be able to inject failure on purpose
to identify areas of improvement and then avoid downtime.
RICHARD SEROTER: Yeah, that's great.
How do you think of doing a chaos experiment
that might span clouds?
Because there's enough chaos that
can happen in a single cloud, in a single region,
in a single AZ.
So all of a sudden, I start thinking about, wow.
I've got services sitting in this cloud, this cloud, maybe
even something on-prem.
First of all, have you designed experiments like that?
What would that even look like?
How do you start?
TAMMY BRYANT: Yeah, definitely.
So a lot of our customers are doing exactly that.
They're working with two or three different cloud
providers.
And so Gremlin is a chaos engineering platform, as well.
And it's really cool to be able to help them
plan for those experiments.
My main tip is always--
I actually started out doing large-scale disaster recovery.
That was my initiation into chaos engineering,
where you take down an entire data center
and failover to a backup one.
And it was more like hot-cold.
But these days, what we would recommend is, hey,
let's actually do something a bit smaller to start.
So let's have a small blast radius and gradually expand.
So for example, if you're using a managed service,
maybe let's just see, what happens
if that service is unavailable?
Does your team know?
Do they get paged?
What if you can't reach it?
And that could be an issue that could happen on your side.
Maybe somebody makes a change, and suddenly, that service
isn't available but nobody realizes.
That's really important.
That's just a small example.
RICHARD SEROTER: That's great.
So it seems like it's a discipline made up
of not just the operational piece, but it's learning.
You're testing out architecture.
You're testing out a lot of things.
What comes out of an experiment?
So when you finish an experiment-- let's
say it's multicloud and you try turning off your cloud
database, you have some things on-prem.
What next?
Typically, what you follow up that experiment with?
TAMMY BRYANT: Yeah.
That's a really common use case for chaos engineering,
exactly what you just described, Richard.
And one of the things that I love to do then
is create patterns.
So a really great thing when moving to the cloud
is, what are the patterns we need
for reliability and performance once we're on the cloud?
And this enables you to actually create those.
And then that's why, if you just move one service first-- that's
my biggest tip is start with one service.
Let's just see how we can do a really great job
with that service.
Create some patterns, and then automate those patterns.
So as you move more services, you
can continuously inject value and understand
that things are still working correctly as you expect.
And then after that, I think one of the really important things
is to share your results.
So what I like to do is write up executive summaries.
This is what our initial hypothesis was.
This is what we learned from injecting the failure.
And this is what we actually went and fixed,
because maybe we had set something up incorrectly
and we needed to make a change for us
to actually be using the cloud in an appropriate way.
RICHARD SEROTER: That's great.
So if someone's thinking about multicloud,
they're thinking about SRE, they're
thinking about chaos engineering,
what should their learning path be?
What's a next thing that you think, let's say, an operations
team might want to learn?
Even a developer-- kind of walk me
through what a good next step would
be for someone who wants to learn more
about this discipline.
TAMMY BRYANT: Definitely.
So for me, actually, when I was working at Dropbox,
we created an SRE apprenticeship program.
I was part of creating that.
And it was very much like a Jedi apprentice program.
I think it is really good to have a one-on-one match
if you want to learn about these types of skills.
And that's just finding a mentor.
Find someone from the industry that's
willing to help you out or maybe someone
from within your own organization
so you can learn about moving to the cloud.
Definitely you can join a Slack community that we run over
at gremlin.com/slack.
There's actually almost 8,000 engineers in there.
And there's a Learning channel, so ping folks there.
But as part of our apprenticeship program,
we focused on having a great mentor,
but also learning by actually just trying things out
in demo environments.
That's a really big tip of mine.
Don't start with production.
Build a demo environment first to learn.
And then actually learn by shadowing.
And then learn by real practice.
So it's training, shadowing, and real practice.
RICHARD SEROTER: That's terrific.
Yeah, hopefully for a lot of folks,
there's a big community here.
Nobody's doing this by themselves.
So Tammy, thank you for joining us today,
and appreciate you taking the time.
TAMMY BRYANT: Thanks so much.
And of course, everyone should read the Google SRE book
if you want to learn more about it, too.
Thank you.
RICHARD SEROTER: Of course.