>> Hey, good news.
We're back in the middle of
a discussion about resiliency in the Cloud.
This time we're going to talk about the core patterns
that make that possible. So join us.
[MUSIC]
>> Welcome to the Azure enablement show,
we will be discussing the challenges that
you and our other texts having customers have encountered.
Together we will be talking with experts to
find out how they think about these problems,
recommended tools, and best practices,
and tips they've learned from
years of experience that you can use.
Today on the Azure we're going to show you
something we've never done before.
We're going to take our script, and we're going to
simply rip it up
because we want to talk today about architecture,
and architecture is discussion.
Hey, so welcome back.
This is a continuation of a previous discussion.
We were talking with Uli and Eric about resiliency,
and we started to get into some
concrete patterns in our architectural panels.
I thought we should just talk about that directly.
I want to get us right back into the conversation.
Eric, when we were last talking about this,
you had a whole list of
concrete architectural patterns to talk about.
Can we dig into those now?
>> Yeah, absolutely. Maybe we
should start with the simplest one that
has probably been intangibly around for a long time.
If you're designing an application,
and it needs to connect to a resource,
whether it's a microservice or a database,
the goal would be if one of those things are unavailable,
then I got to retry.
I can't just try once, give up,
and then return a failure message to the user.
The goal would be,
I'm going to implement a loop of some sort to say,
well, the database is not available, how about now?
But I think that this might
actually be dangerous so you can't really
implement these things myopically, and in a vacuum.
Sometimes you have to stack them in your implementation.
>> Yeah, Eric, that's a really good point,
especially when you're on the Cloud.
Cloud services protect themselves from abuse effectively,
and we call that throttling.
It's really an important part of the Cloud architecture.
If you deploy or develop a service that you want to expose,
you should really think about throttling
architectures in your service as well.
Which means if there are too many requests coming too fast,
the service starts to effectively
discard those requests because it's trying to stay up,
and running, and be available.
As a person that implements the retry pattern,
don't increase the frequency if you're failing.
We've seen unfortunately a lot of code that does
exactly that because they think
that asking more frequently for a connection makes it better.
It's actually the opposite.
The first time you do a 30-second interval,
the second time maybe you do 45 seconds,
and then you might want to give it
the five-minute break to see if
the service you're dependent on recovers.
There's other patterns that then come into play,
where for example, if your database isn't available right now,
hopefully, you have built a cache of the data
you need to display in the pages and so forth,
so that the user interaction can keep
going while you in the background are
trying to retry the connection
so that you get to the latest and greatest content.
I think there's a couple of
patterns that you could almost put into a "pipeline",
where if you start with retry,
the service uses throttling to protect itself,
and then you effectively add caching as a pattern so that
you have something to show to
the user that effectively keeps the system going.
>> Well, let's talk about caching
a little bit because I think that is the next important pattern.
Caching, many of us think that that's done for performance.
Like I actually want to hit my cache when I have a high cache
hit ratio before I make an expensive call to the slow path,
which might be the Cloud or
the database back-end, and my data-center.
But caching to me,
actually has multiple dimensions.
But, what Uli was talking about, is user-experience caching.
If my app is disconnected expectedly or unexpectedly,
I need to hit the cache for performance reasons,
or to cover up some type of failure that the app depends on.
But, cache also has non fonts,
so that's more of a functional capability for caching.
But cache also has a non-functional capability,
and this is what Uli was talking about in one of our other videos,
which is maybe I actually cache
the hard IPN points of my back-end services so when DNS fails,
I can still make a connection to my back-end.
>> The other piece I want to add to the caching pattern,
a lot of people think a cache is a cache,
and technically speaking, that's correct.
The important thing is that you need to understand the details
of the attributes of the data you're trying to
cache because not everything can be cached.
Let me explain a little bit.
When you're looking at a catalog,
most of you don't even know a physical catalog anymore.
But, when Sears was very popular,
they used to send out catalogs in printed form,
and that catalog is actually reference data in
the larger sense of the Word because the Sears guys,
as soon as you call them on the phone they would say,
"Hey, which version of the catalog did you use?"
They would allow you to use up to
two or three catalog versions
before they ask you to say no, sorry,
we can't take this order anymore,
we will send you a new one because
the reference data that was in that catalog was outdated.
You need to really understand
the property of the data you're putting into the cache.
Reference data is something that is effectively cut off.
The database, can be marshaled around,
copied and so forth because it's Read-only,
it has a fixed attributes so it's perfect for caching.
There are other pieces of data,
like the shared inventory that you have.
You don't want to put it into a cache because a cache doesn't have
the properties of a database that allows
you asset transactions and so forth.
Thinking through your data that you want to put in
the cache is really important as part of this pattern,
and there's great documentation out there.
One of my favorite papers is from
an ex-Microsoft person called [inaudible] ,
and he wrote a paper in 2003 I think of data on the inside,
and data on the outside.
That was in the context of service-oriented architecture.
But, there is a treaty in there around what data,
like reference data and so forth,
is that I highly recommend any architect,
understand and consume,
and then apply to their problem as best as is feasible.
>> Okay, so we have so far retries,
caching, and throttling.
I also heard something go by about circuits.
What's a circuit breaker pattern really?
>> Well, David, it's actually a way for
an application to protect.
[inaudible] say, look, I have now encountered
so many errors from this database or whatever resource I'm
trying to use and I'm okay with
a certain threshold on errors
because I expect nothing goes perfect.
That would be awesome but generally, in life nothing does.
I count the number of
errors and I start to see they are too many of them,
and so what I'm doing is I'm throwing
this circuit breaker which protects
my application from this rogue service
or this service that doesn't work,
and I'm indicating that I want to go down a different path.
For example, Eric mentioned the retry pattern,
you keep on retrying but at some point in
time you just have to say, the service is dead.
The circuit breaker will then say,
go let down the other path and the other path could
be there as a secondary service you've set up.
There's a backup database, for example,
that you use to do this or you have a cache,
where you normally don't go because you
want to have the last and up-to-date information.
But in the case of the circuit breaker of triggering,
you might want to go to the cache so that you can survive,
like the Netflix example, we used in the other show episode.
The circuit breaker is really
indicating to the application it's time to
stop what you did here because it's not
going to get you anywhere and move down a different path.
>> I think it can be from both sides,
the resource itself can
implement the circuit breaker pattern to detect
a bad or rogue application
that might be overwhelming it with calls.
It'll say, I'm going to shut you down because
you now are exhibiting bad behavior,
which is going to compromise my ability to answer
all of the good citizens that are asking for my resources.
>> Yeah, I think it's very
interesting because I had heard of this as
a protective measure on the server-side,
the server doing its best to keep itself alive
and it's like a submarine where
you make sure you compartmentalize things sufficiently,
you close the bulkhead.
Maybe that's a bulkhead pattern. Is that another pattern?
>> There's another called the Bulkhead pattern.
>> Oh, yeah. Okay. What's the Bulkhead pattern?
Maybe that's where I'm getting confused here.
>> No, the bulkhead pattern,
the way I think about it is the bulkhead pattern
is just like a ship with multiple components as you said, David.
The goal would be, if the water gets into the first compartment,
it's not going to drag down the whole ship because there are,
I'm not a ship builder, but there are compartments or
bulkheads between the sensitive areas of the vessel.
Think about applying that paradigm to
an application where things run in pools.
If an application component becomes jeopardized,
it's the pool that gets recycled or
restarted while other pools continue their functionality.
Its bad behavior around a specific pool of resources gets
contained and it doesn't spread or spill over
into the other areas of functionality of the application.
>> Again, David, sorry.
>> Yes.
>> If you're looking at the bulkhead pattern,
it actually gets coupled very nicely with some
design-thinking that I think everybody should go through.
What Eric was focusing on was the resources,
like the database pools and
other things, but there are other things.
When you look at our first episode,
we talked about how Netflix
separates the catalog from the recommendations,
from the actual streaming itself.
For me, that's also bulkheads
because you're starting to looking at a number
of application working loads
because every application has multiple workloads.
For me, the first step an architect needs to do is to
understand the application scenario
end-to-end and then start to say,
okay, how do I partition my system so that I can
apply different high availability requirements or other things?
The worst thing I think an architect can
do especially in the Cloud world,
is apply a blanket set of patterns and technologies
to the entire solution because
that's going to be more expensive than it needs to be.
Now with Cloud,
you can actually pick and choose what makes
sense for the specific scenario that you're looking at.
Your all app application is not a monolithic thing anymore.
Again, Eric mentioned microservices
as one way of implementing it, but for me,
the modeling of the application is a key activity that then gets
augmented with the implementation like
microservices and the patterns that we have mentioned so far.
>> This sounds like pretty good advice.
I think we're going to run out of time on this particular episode.
Do either of you have 30 seconds of a piece of advice you want to
give people about designing
Cloud applications for failure, to wrap this up?
>> Yeah, I think focusing on what the experience is going to be,
is going to be crucial.
It will dictate what type of
application design practices that you use
and what failure and resiliency patterns that you employ.
I think thinking about Uli's point,
there's functional and non-functional
implementations of these resiliency patterns.
Where I was talking about connection pooling and resource pooling,
he was talking about functional segmentations of
capabilities and being able to
have the bulkhead pattern applied to those.
Think about things functionally,
from the infrastructure level,
from a Cloud perspective and a microservice point of view.
Think about things from a functional level,
which would be the apps and
their domain and separation of concerns,
and then implement the patterns that will
feed and support the desired user experience,
both in failure and non-failure states.
>> Yeah, the only thing I would add here, Eric,
is in the Cloud world,
we're living with two different life cycles.
In the on-premise world,
what happens is,
your application gets deployed and
then it's tightly managed from a change management perspective.
Infrastructure, your changes a very
managed together and so that disruption is minimum.
In the Cloud, the Cloud provider will continue to make changes,
enhancements to the services you rely on and in the general case,
nothing happens, you don't know this.
But you need to be prepared that some of
these changes effectively leads to problems in
your application and you can't ask the Cloud provider to
synchronize with your change management process because again,
you are just one of hundreds of thousands
of customers running on that Cloud.
You need to think through how you live or survive
Cloud failures or changes in the Cloud service that you
rely on and come, how do I say this?
Recollect your application and will continue to run.
That's a new thing that the Cloud has really
introduced that I think a lot of people struggle with at first,
but there are great news that there are
patterns out there that can help you with that as well.
>> Maybe David we should do an episode
on chaos engineering as a practice?
>> Oh yeah, I'm definitely up for that.
Let's do that in the future episode, I think.
But I think this is a really good place to stop.
Thank you both. Look forward to
talking with you some more on some other topics.
>> Awesome. Thank you, David.
>> Thanks, David.
>> That was a really cool set of conversations,
I hope you get a chance to watch all those episodes.
We're going to move on to another topic here on
Armchair Architects as part of the Azure Enablement Show.
[MUSIC]