SPEAKER: If you are ingesting large amount of data
for analysis, or looking to simplify
the development of event-driven microservices,
then you need to check out Cloud Pub/Sub.
Pub/Sub Made Easy is a series dedicated
to helping you understand Cloud Pub/Sub inside out.
So let's get started by going through an overview today.
[MUSIC PLAYING]
Adam and I have been collaborating on a project,
but we are based in different parts of the world.
I'm here in California.
And he is in Australia.
So I just do my part on the Google Doc
and assign it to him for review.
And he does the same.
This allows us to receive messages and act
on them in our own working hours,
essentially asynchronously.
We don't have to be available at the same time
and are still able to get the project done.
Cloud Pub/Sub is based on the same concept
but for applications and services.
Pub/Sub helps build robust, scalable systems
of applications by integrating them asynchronously.
Cat-Out is a website to purchase fun cat posters.
When a user orders a pretty poster,
Cat-Out packages the order, ships it to the user,
and notifies them.
In a very simplistic service oriented architecture,
they will need at least four microservices--
a packaging service to pack the poster, a shipping
service to generate labels, and a notification service to send
notification to the user about the status of their org,
all these tied to Order Manager Service.
Granted that there'll be some payment services, a reporting
service, et cetera.
But for the purpose of this conversation,
let's assume those all exist.
Now, let's trace what happens when a user orders a piece
of art.
The order service receives the order
and sends a message to the packaging service.
The packaging service then does the necessary processing
to package the poster and sends a message to the shipping
service to ship it.
The shipping service does the necessary processing
to ship the package and sends a message
to the notification service, which notifies the customer
that an order is processed.
In the same way, the order and packaging service
also send messages to the Notification service
to send to the customer the updated
information on their order.
There are some problems with this arrangement though.
One, it's just complicated.
It's a lot of arrows.
But as we drew deeper, what happens if a service, say
the packaging service, fails?
Well, we need some kind of retry mechanism
that would keep the whole system from banking.
Hm.
Can be done, but a lot of work.
What happens if a new service gets introduced
like a monitoring service that logs all of these transactions
to a central store?
This service needs to interface with all the other services,
which will also require making a change in all
the other services.
That makes the developers work really hard.
Think about testing.
That would be hard too.
We will need mock services to test
each of the individual service.
Let's say I need to test my packaging service.
Because it is dependent on the shipping, order,
and notification service, I need to mock up
all of those services just to test my packaging service,
which is a lot of work.
What if you want to scale this system?
If there are three instances of packaging service,
there would be a load balancer involved.
You would not know how to cancel the order, which
stage it is in, and can it still be canceled.
This is where Cloud Pub/Sub comes to rescue.
Cloud Pub/Sub is a fully managed, real time messaging
service that allows you to send and receive
messages between independent applications.
Once an event is successfully published as a message,
it becomes the job of the service
to ensure that all the systems that need
to react to this event get it.
The idea behind asynchronous integrations
is to react to events represented as messages.
In our example, messages merely represent
events happening in the real world like order received,
package shipped, et cetera.
Let's see Cat-Out's flow of events with Cloud Pub/Sub now.
The order service, instead of sending messages directly
to the packaging service, will just
send these messages to Pub/Sub where they will be stored.
The packaging service and the Notification Service
will receive these messages either by a push or a pull.
Packaging service, instead of directly sending messages
to shipping service, would send them to Pub/Sub.
And the event will be received by the shipping
and notification service.
The packaging service did not need
to know which services are receiving this event.
This makes it simple.
Similarly, the shipping service sends the messages to Pub/Sub
from where the order and the Notification Service
pull or receive those messages.
By using Cloud Pub/Sub, we've solved a lot of problems
that we had in the earlier architecture.
Well, one, we've made each piece simpler
since it only needs to emit an event without worrying
about who gets it or receive an event without much care
about how it got there.
But more importantly, this is more scalable.
It reduces dependencies.
Services can change and don't need
to know about each other's changes.
All that need to be made sure is that they
are subscribing to Pub/Sub and receiving the right messages.
It's also much easier now to introduce new services.
If our new monitoring service pops up, all we need to do
is make sure it subscribes to the messages
from the existing service via Cloud Pub/Sub.
The architecture with Cloud Pub/Sub
has built in resilience.
If any service were to break down,
Pub/Sub will store the messages for seven days
or until the service comes back up, and it's delivered then.
And because Pub/Sub is fully managed service,
you don't have to worry about it being single point of failure.
It's scalable, and resilient, and can handle millions
of messages simultaneously.
All right.
So today we learned when to use Cloud
Pub/Sub and the common challenges it solves.
Join us next time to learn how it works
and some of the most common use cases.
If you like this video and would like to see more such content,
don't forget to like and subscribe to our channel.