PRIYANKA VERGADIA: Cat-Out has decided
to set up their packaging subscriber on Cloud Functions.
Stay tuned to find out how.
So far in the series, we learned about Cloud Pub/Sub
and how it works using Cat-Out's Asynchronous Order Processing
Service.
We saw how Pub/Sub helps manage the interactions
between the order, packaging, shipping, and notification
services.
In the sixth episode, we learned that Cloud Pub/Sub
supports both push and pull message delivery.
Quick recap.
You use pull if you don't have public HTTPS endpoints, which
means your service is private or firewalls are involved.
Pull is also more efficient and minimizes latency.
So if events are generated at a really high rate,
like hundreds or thousands of events per second,
then pull is the way to go.
Also, push delivery requires the client endpoint is a publicly
accessible HTTPS server.
Luckily, this is easy to set up with Cloud Functions.
Today, let's dive a bit deeper into
push subscribers for Cat-Out.
Setting up Pub/Sub to send messages
to a client using push delivery is extremely easy.
It requires no special client libraries, just
a standard HTTPS endpoint routable
by DNS and a web server to respond to these requests.
It's also a lightweight way of programming,
particularly with web server frameworks
such as Cloud Functions, Cloud Run, and App Engine.
You simply write the server code and let the framework
take care of the communication.
Of course, you can choose to host this application
on a virtual machine anywhere you like.
It's usually more effort, but if that's something
you need to do.
But our friends at Cat-Out are all in
for the lightweight programming.
So they decided to use Cloud Functions for the packaging
subscriber that receives messages
from the order service.
That Pub/Sub push request coming to the packaging subscriber
would be HTTPS post.
That looks like this, where the data field is base64 encoded,
and the subscription is defined.
Once processed, the packaging subscriber
sends a success response, like 200, 201, 202, 204, and 102
are interpreted as message acknowledgments
by the Cloud Pub/Sub system.
Pub/Sub does not send a negative acknowledgment.
So if the packaging webhook does not return a success code
within the default acknowledgment deadline,
then Pub/Sub retries delivery until the message
expires after the subscription's message retention period.
Oh, and the cool part, the rate of push delivery
is automatically adjusted to maximize delivery rate
while not overwhelming the push endpoint.
This is accomplished using a slow-start algorithm.
Let's see how this works.
The system starts by sending a single message at a time.
With each successful delivery, the number of messages sent
is doubled.
And the rate continues to double until there is a delivery
failure or the system reaches a quota or resource limits.
For each delivery failure, the number
of concurrent requests to the endpoint
halves until a minimum of one request is reached.
So this algorithm assumes that there
are enough published messages in the queue
to sustain this throughput.
Ultimately, the push delivery rate
is constrained by the rate at which messages are published.
We understand the basics of how a message is structured
and the slow-start algorithm works.
But we did not discuss the most important piece of the puzzle--
how does Cat-Out authenticate and authorize Pub/Sub
to make a call to the subscriber endpoint?
There are a few options.
Let's see.
They could just set up the packaging subscriber
with no authentication, which is not a really good option
for a private service.
So the real first option for authentication
is that the publisher sends a custom authorization
token as a part of the message.
The subscriber then authenticates that token.
The second authentication option is
to configure the push subscription
to include a signed OpenID Connect
token for a GCP service account in the authorization header.
And the push endpoint can use this token
to validate that the request is issued on behalf of the service
account associated with the subscription
and make an authorization decision.
We know that Cat-Out has chosen to set up
Cloud Function as the packaging subscriber.
With Cloud Function, the authentication
and authorization is even easier,
because it uses the standard IAM permissions.
Your code just handles the message
and attributes while Function manages the request, parsing,
and responding to Pub/Sub
To find out how simple it is, let's set this up
and see for ourselves.
So let's create a Pub/Sub topic and then create a Cloud
Function in the same project by clicking
on the Trigger Cloud Function button right from the topic
itself.
It autopopulates the trigger as Cloud Pub/Sub, the topic,
and the function name.
Once the function is created, we go to our topic
and publish a message.
Now it is time to check if the message was
received by our Cloud Function subscriber.
We navigate to Logs, filter on Cloud Function,
and see that our message is received.
Now, you may ask, what if Cat-Out's packaging subscriber
has to go in maintenance mode for some time?
In temporary situations like that,
it is recommended to change the subscription to pull.
And when the service is ready to take requests,
just set the URL to a valid endpoint again.
If Cat-Out decides to change the packaging subscriber
to a different service altogether,
then they would have to permanently stop delivery
to the old endpoint.
In such cases, it is recommended to just delete
the subscription, create a new one,
and add the new subscriber to it.
All right.
So today we learned that push subscriber is a simple HTTPS
endpoint.
We learned the different authentication authorization
mechanisms and saw how easy it is to set up a push
subscriber on Cloud Functions.
Stay tuned for the next episode, where
we will help Cat-Out set up a subscriber
endpoint on Cloud Run.
Until then, give us a like, subscribe,
and let us know what you want to learn about Cloud Pub/Sub.