PRIYANKA VERGADIA: If you are ingesting large amount of data
for analysis or looking to simplify
the development of event driven applications,
then you need to check out Cloud Pub/Sub.
[MUSIC PLAYING]
In the last couple episodes, we learned about Cloud Pub/Sub
through Cat-Out's asynchronous order processing website.
We saw how Pub/Sub helps manage the interactions between order,
packaging, shipping, and notification services.
In today's episode, let's build a Cloud Pub/Sub system
for Cat-Out.
As seen in the last episode for Cat-Out's application,
the whole system will comprise of three topics corresponding
to order, packaging, and shipping service
for the sake of understanding.
Let's only consider the order service sending messages
to packaging and notification service
and build that part of the system.
Feel free to extend this further and build the whole flow.
All right, so we will have a publisher application,
Order Service, which sends a simple message, Order Received.
This message gets sent to two subscribers-- the packaging
service and the notification service, which
process the message.
And then these services process and send the message
forward to their topics and subscribers
to complete the order flow and process to order fully.
OK, so now that we know what we are building,
we are going to build this in four steps.
Let's start with step one where we
will create the Cloud Pub/Sub topic for the order service
and the two subscriptions for packaging and notification
service.
If you are new to Google Cloud, create a free account
by navigating to cloud.google.com/free
and following the instructions there.
Then navigate to the console and create or use
an existing project.
Make note of the project ID.
Now, let's navigate to the Cloud Pub/Sub
subsection of the console and click on Create a Topic.
Let's call this topic order_topic.
Our order service, which is our publisher,
will send messages to this topic.
Then scroll down on the Create Topic
and click on Create Subscription.
Let's name the first subscription as package_sub.
Leave the rest as default, which will
create a pool subscription.
We use the same procedure to create
another subscription attached to the order topic called
notification sub.
At this point, your Cloud Pub/Sub environment
is ready with the order topic and the two subscriptions.
Now, we are onto step two where we
will create a service account that the applications use
for authentication and set up Cloud IAM permissions.
Navigate to Service Account Section of the console
and click Create Service Account, enter a service
account name, such as Pub/Sub System, and click Create.
For this example, the service account
needs publishing and subscribing permissions.
So we use the select a [? rule ?] dropdown to add
the Cloud Pub/Sub Publisher and Cloud Pub/Sub Subscriber
[? rule. ?] Click Continue.
And we do not need to grant user access to this service account.
Then create key.
This key is used by the Client Library
to access the Cloud Pub/Sub API.
Select JSON and click Create.
The key is sent to the Downloads Folder.
For the purpose of this demo, we can leave it there and rename
the key file to key.JSON.
Now, the next step is to start the three
independent applications.
One, publisher, which is our order service,
and two subscribers, which are our packaging and notification
service.
For this step, we need the Google Cloud SDK installed.
If you don't have it, today's the day to download it.
If you're not already in the Pub/Sub project
on Google Cloud SDK, then set the project using
G Cloud Config set project.
List the topics to see the order topic
we created from the console and list subscriptions
to see the packaging and notifications subscriptions.
If you don't have Python, this would be a great point
to install Python and set up virtual environment using
the steps in the description below.
Create a folder where you want to download the sample Python
files for this demo.
cd into it and download the code.
Now, open three new terminals.
One for our order publisher and two for our subscribers.
In these terminals, create a Python virtual environment
and activate it.
Upgrade the Google Cloud Pub/Sub in all three terminals
and associate the JSON key with the service account created
earlier.
Now set up an environment variable
with our current project ID.
We use the G Cloud command to determine currently selected
project ID and set it as the project variable.
Verify that the project variable is set to our current project
ID.
cd into the directory with sample code
and get into our publisher file, pub.py.
Change the message from hello world to order received.
Now, we are ready to send messages.
Let's start our packaging subscriber now,
which is Listening and our order publisher.
Our order application sent an order received message
with a message ID while being unaware
of any existing subscriptions.
The packaging subscriber application
receives the order received message and prints it
and sends an acknowledgment to Cloud Pub/Sub.
Pub/Sub receives this acknowledgment,
which indicates that the message has been processed successfully
and does not need to be sent to the packaging subscriber.
Cloud Pub/Sub then removes the message
from that subscriber backlog.
Now, let's start the notification subscriber.
And as soon as it is started, it receives the message
published by the order topic.
The difference here is that this subscriber
isn't started until after the publisher has sent the message
to the topic and subscriptions.
If the notification service was called directly from the order
service, then the message would either
have to wait a long time or time out
since the service was not available.
Cloud Pub/Sub manages this process
by effectively saving the message for the order
notification subscription.
Today, we built a simple Cloud Pub/Sub system
and learned how it sends messages
from a publisher to a subscriber.
At this point, you should be able to build
a simple application with Cloud Pub/Sub.
In the upcoming videos, we will cover more important details
and choices you will need to make when building a real world
application.
That's all for today on Pub/Sub Made Easy.
Join us next time to check out a deep dive into publishers.
If you like this video and would like to see more such content,
don't forget to like and subscribe to our channel.
[MUSIC PLAYING]