Hi, and welcome to another episode of 'Back To Basics'.
My name is Derek,
and I'm here today to talk to you about, transaction patterns for distributed applications.
So let's start off by understanding what we mean
when we talk about transactions.
Well, a transaction is simply a group of actions
that are performed as if they were a single bulk action.
For example,
if you want to order goods in an application shopping cart,
or you want to place a food order in a food delivery application,
or even book your next flight overseas,
all of these examples are considered transactions in an application.
Let's first look at how a transaction would look
inside a traditional non-distributed monolithic application.
Let's do this by taking a food ordering scenario
inside a food delivery application.
The user would initiate the transaction with a call to order food
from the application from end.
This call would then be received by the application's backend.
There may be a number of actions
that make up the order food transaction.
Actions like, order.
This action would be where a record of the order would be created.
And the details of the order persisted.
Pay.
This action would cover the processing of the payment,
and the updating of the payment status.
It might also use a third party payment gateway.
And finally, deliver.
This action would be where the delivery details would be updated,
and sent to the delivery driver.
As each of these actions completes,
they would in turn update the overall transaction state in the single database
before calling the mixed action in the transaction sequence.
If however, one of the actions did not complete,
then they must also roll back the changes the preceding actions have made
on the single database.
This type of transaction sequence and its flow inside the monolithic application
usually has the undesired effect of making each action in the transaction
tightly coupled to each other and to the underlying data store.
In modern distributed applications,
it is best practice to have these actions each in their own separate service.
And each of these services would have their own individual data store.
This best practice approach reduces issues around coupling,
while also meaning the services themselves can be built and run
on different platforms, services, and technologies.
For example, here we have our three services:
order, pay, and deliver,
all running on three different AWS services.
In this case, we have order and pay
running inside containers on Amazon Elastic Container Service or ECS
and then deliver service using AWS Lambda and Amazon DynamoDB.
So how do we guarantee that transactions running across these services
behave like a single bulk update,
and that data consistency is maintained across these microservices
and data stores that could be running anywhere and on anything.
Well, an antipattern that we might see people used to solve this problem,
might be to chain each microservice update
that is part of the transaction together.
Very much like what we would do in a monolithic example.
This would mean our food ordering scenario might look something like this.
When an order request is made,
the order service processes the order,
updates its own database,
and then makes a call to the payment service
to process the payment.
If the payment succeeds,
the payment service calls the delivery service
to try and arrange delivery of the food.
Once the delivery service has successfully arranged delivery,
it will then call back to the payment service,
which will then in turn call back to the order service to notify the user.
This pattern may seem fine
however, there are flaws in this approach.
which are generally caused by the assumption,
that nothing will ever fail when each service
is communicating with its peers.
What we actually see,
is failures are part of distributed computing
and must always be taken into account.
They may occur because the underlying network
is unreliable or the bandwidth
that is available is being consumed by other competing operations
or latency between the services slows calls down.
Because with distributed applications there's no guarantee
that the service involves are running in the same room,
country, or even continent.
One or more of these could impact our food ordering application
and make the overall transaction state hard to determine.
So this time the user may call the order service as always,
which is successful and in turn calls the payment service.
However, due to network latency,
between the two services
or a delay inside the payment service itself,
a timeout then occurs inside the order service,
which then rolls back the order
and lets the user know that no food has been ordered.
However, what actually happens is,
after a delay, the payment service eventually processes the payment.
This delay may occur
because the third party payment getaway is slow.
The payment service then calls the delivery service,
and the food is set to be delivered.
But remember, the user thinks the order has failed
and we will get quite a shock when their food turns up.
Not a good look for this application.
So how can we prevent this from happening?
Well, we could put in a retry pattern for all the inter-service calls
but retry calls between services a certain number of times.
However, putting retries would not fix all the issues
and could in fact introduce some of its own.
Putting in indefinite retries may actually mean
the applications users with wait indefinitely, for some orders to be confirmed.
What we need ideally, is a service that controls the flow
and data consistency of each transaction inside each service.
Maintaining the current state of the transaction,
and having the ability to roll back or compensate
for a fail transaction in each service.
Luckily, there is a pattern that we can use to do just that
called the Saga pattern with orchestration.
With this pattern, you define an orchestrator
or transaction manager
as a service that exists only to facilitate
all the service transactions or Saga.
It is responsible for coordinating the global transaction flow,
that is communicating with the appropriate services
that are involved in a transaction
and orchestrating the necessary rollback
or what can also be referred to as a compensating action.
The Saga pattern therefore removes the need for each service
to have any knowledge of other services
that are involved in the transaction.
We can implement a service orchestrator via Saga,
by using AWS Step Functions to control and maintain
state of the global transaction flow.
Step Functions can integrate with a large number of AWS services
that could be running the services
that the Step Function services is orchestrating.
They could be running inside AWS Lambda,
Elastic Container Service, as well as Elastic Kubernetes Service,
or even behind an API Gateway to name but a few.
So in our case, a Step Functions workflow might look like this.
We would have our place order, make payment, and deliver order
as part of the workflow.
Each of these steps would call out to the underlying service
and wait for that service to respond with a success or failure.
We can also build in a suitable retry pattern
to each of these calls, from within a Step Functions workflow.
Should the service return a failure,
or all the retry become exhausted?
We can also add the rollback or compensating actions into the workflow.
So it looks a bit more like this,
as we add in the compensating transaction flow.
The primary advantage of the Saga or orchestrator pattern,
is that the complexity is somewhat centralized into one component.
So the other microservices in the Saga,
don't need to even be aware of the Saga
or the other microservices it interacts with.
And AWS Step Functions, is a great way to visually represent the Saga
so it makes it easy to debug.
If you want to learn more about these topics,
check out the links below.
Thanks for watching this episode of 'Back to Basis', we'll see you next time.