SPEAKER: Are you ingesting large amount of data for analysis
or looking to simplify the development
of event-driven applications?
Then stay tuned to learn more about Cloud Pub/Sub.
In the last episode, we looked at Cat-Out's asynchronous order
processing service with and without Cloud pub/sub.
We also learned how Cloud Pub/Sub
is perfect for such use cases to scale seamlessly
for millions of messages.
Today, we will consider our Cat-Out example again,
which is a simple case of a workflow
application implemented as a set of services connected
asynchronously by messages.
We will also see how Cloud Pub/Sub helps Cat-Out expand
their application globally.
As the name Pub/Sub indicates, Cloud Pub/Sub
supports a publisher subscriber model.
A publisher application creates and sends messages
to a topic which is a named resource.
And the messages are stored until acknowledged
by all subscribers.
To receive these messages, a subscriber application
creates a subscription to a topic.
The subscriber receives a message
by either Cloud Pub/Sub pushing them
to the subscribers chosen endpoint
or by subscriber pulling them from the service.
When the message is acknowledged by the subscriber,
it is removed from the subscription backlog
and not delivered again.
The message itself can contain the payload and optional
attributes that describe the payload content.
Communication can be one too many also called
fan out, many to one or fan in, and many to many,
which means if subscriber x wants to receive messages
from two different publishers, a and b,
it can be achieved by creating two subscriptions corresponding
to the topics for each publisher, which
is a many-to-one model.
Similarly, if two different applications
want to subscribe to the same message from a publisher,
then it can be achieved by creating
two different subscriptions from the same topic, which
is a one-to-many model.
Now, remember Cat-Out, the order processing and delivery service
from the last episode?
Let's see how Pub/Sub topic and subscriptions
would be set up for this asynchronous functionality.
The order service sends the messages to Pub/Sub topic
where they get stored.
The packaging and notification service
subscribe to these messages through the respective Pub/Sub
subscription in a fanout fashion,
Packaging service will send messages to Pub/Sub topic.
And the shipping and notification service
will subscribe to the respective Pub/Sub subscription.
Similarly, the shipping service will send the messages
to Pub/Sub topic.
And the order and notification service
will subscribe to their respective Pub/Sub
subscriptions.
Now, Cat-Out is going to a global expansion
with packaging service in Europe and order processing
service in US.
Because Cloud Pub/Sub is a global service,
it ensures that messages get from one location
to another no matter where these services are
with consistent latency and no replication necessary.
This is huge for the Cat-Out team,
because they can focus on developing the service
and leave the global scaling to Cloud Pub/Sub.
What do we need to know about publisher and subscriber
endpoints?
Well, let's see.
Publishers can be any application
that can make https request to Google APIs.com.
An App Engine app, a web service hosted on Google Compute Engine
or any other third-party network and install an app
for desktop or mobile devices or even a browser.
Pull subscribers can also be any application
that can make https requests to Google APIs.com,
although Push subscribers must be web hook end points that
can accept Post requests over https.
Cloud Pub/Sub is all about service integration.
Data is ingested into Cloud Pub/Sub
from the front-end services databases and other event
producers, then received by Cloud Dataflow
or our other event processing service
for de-duplication, ordering, and enrichment.
From Cloud Dataflow, the event can
be sent to multiple targets.
It can be used to facilitate end user experiences,
like managing notification via Firebase notifications
and messaging, or storing end user state in Firestore
or managing an IT device.
Other targets could be data warehouse or data lakes,
like BigQuery and GCS or event-driven and [? sololess ?]
apps, like Cloud Functions, App Engine, and Cloud Run.
Bigtable could also be your target
for operational databases or just caches.
Cloud Pub/Sub has lots of different use
cases, which can be categorized into two broad buckets.
It's great for streaming analytics or ingestion of data
into analytical systems.
It's also great for implementing asynchronous workflows,
like the order processing application
we saw earlier in the Cat-Out example.
Let us know how you're use Cloud Pub/Sub in your environments
in the comments below.
All right.
So today, we learned what is Cloud Pub/Sub, how it works,
and some of the most common use cases.
Join us next to dive deeper and actually build a Cloud Pub/Sub
system from scratch.
If you liked this video and would
like to see more such content, don't
forget to like and subscribe to our channel.