Hello, my name is Jamil Spain,
Developer Advocate with IBM Cloud,
and my topic for today is
Message Queuing.
Now, as a technologist, when I first
started out developing applications,
there was an expectation that in my
code that applications would
process everything immediately.
And even when I first started
dealing with REST APIs, there's
really the expectation that I make a
front end, makes a rest call to a
backend through to a REST API call
that you're expecting a response to
come back. Some type of response
code must go back to let me know
404 was not good, 200
it was good.
But again, there's that expectation
that things happen instantly, and I
get instant confirmation that
something is happening.
And often I wrote a lot
of code to
accommodate situations where
that communication could not happen
or did not
function properly there.
But today's topic kind of branches
out through that.
Let's deal with break down what the
term message queuing actually means
here. So overall, from a
thirty thousand foot view,
it's a definitely a architecture
technique that allows you to
break applications apart.
Apps apart.
With what we like to call
asynchronous communication.
I'll abbreviate that there.
So asynchronous communication.
So I want my particular
applications to or particular
subsets to still function
and not worry about the other parts
there versus the
other part where all my code
was in one stack, and
in only in a monolithic application
or when I built an application where
all the functionality exists
that, you know, one
set of functions and one set of code
on my application.
I'm just always handling a
transaction off to another set of
code. But there is an expectation
that things follow through and
process right then.
There's no way to really kind of
delay things to happen later.
And this is what message queuing is
all about. Well, let's break down
the first part here.
First, let's deal with the
word message.
And as it did denotes that
it is really something
that a piece of
information that wants to get
processed from one system to
another, an application
will just say subsystem for me as
well, kind of denoted by
with that particular piece.
So you have data
that needs to get
transmitted. Now this could be
actual data payloads, actual
files, it could
be metadata that that reference
is that something should happen on
the other in.
So many different types
of data
formats can be there.
All right. And then we have the word
queue.
Which queue in the most simplest
form refers to kind
of a line of things that a process
sequentially.
So when you think of a message
queue, you think of the messages
of coming into a queue
as a way that kind of holds the
messages in the order they are
received and delivers them
to the actual party.
So one way to think about this,
I know this kind of explained
message, explained queuing,
explained the overall, but think
about a tool that everyone uses
every day, email.
Email is an excellent example
of asynchronous communication
and how that works.
I can, theoretically or
not, theoretically, actually in
reality, in real life,
I send a message out, but
I can wait for a response later.
It goes into an inbox.
Think of that inbox, your email
inbox as a queue of messages
that are held there.
And at some point the
person is going to receive it
has the opportunity to look at it
whenever they want, if they take a
longer time, there's a larger list
that has accumulated of messages
that you have to read, which is
usually the case when I go through
some of my email email boxes,
or for ones that I check constantly,
like, say, for instance, your work
email, you always constantly
going through the list that you're
there waiting to kind of reply
back. But excellent example
of asynchronous.
I can really fill out an email,
compose it, send it off.
I'm not worried about whether you
received it or when you answer it,
but I can move on to something else.
All right, so perfect example of
asynchronous, and we'll carry that
example through as
well to some of the use cases
here.
So we talk through those,
let's talk about the type of
messages that occur.
So we've mentioned queue, which is a
line
of messages.
So if I were to draw this out
in this perspective here, let's do
it this way, let's
take this green as the queue.
And they also say that each one of
these is going to be an actual
message.
And so at some point you're going to
have the...
They're going to send the kind of
the message here.
And then at the other end,
somebody is going to open it up.
Get a little envelope there.
All right, a little artwork here.
So the person that's putting the
message on is going to be what's
known as the producer.
And then the person consuming the
message is the consumer.
So these are often the two profiles
or roles that you can take.
And of course, you can be the
producer one time and be the
consumer on another example.
So this one pattern here that you're
looking at is more like a
point to point.
I am setting it for one application
to the other.
All right.
It could be a
let's say I have a web form
and you're filling out, and as you
complete the web form, I take the
metadata from that form, JSON
object, I could put that on a queue
to be processed later to be inserted
into a database, to be
inserted to another system for
marketing automation, whatever
it may be the use case.
The point is that I have, I need
messages to guarantee delivery to
get there and not
be dependent upon you, submit
the form, I send it back to that
other system and get a response
back.
I can keep ingesting particular
messages, forms and middles and
keep loading those in to go from
there. So when you have these kind
of use cases, these are excellent
things. They should read out that
hey, instead of me just doing direct
http connection, let me think of a
message queuing pattern that
I could facilitate to go here.
So we'll call this point to point.
The next one is something
like a publish, publish
subscribe.
Also abbreviated as PubSub
that you will hear and this works
from the capability that I have
a producer.
Let's go back to the green, I'll do
another queue, and we'll do
more messages that come in.
So on the PubSub model,
I'm going to have
multiple
parties or applications that
are interested in getting
this message
as it comes through.
All right, so pub, publish
subscribe, so these consumers
are listening for this particular,
I'll call it a channel or interested
in this particular thread of
messages that may come across.
So it could be that new customer
signs up, you may want to send
something to
for a new order, eCommerce order
comes in. You may want to send
something to accounting for an
invoice to go out.
You may want to send something to
the shipping or notifications
for stock or, you know,
any multiple things
that you want to kind of get
through. That could be something you
can shoot out to kind of go.
And the last one I'll work through
is what we like to call
request-reply reply-requests.
It's done from the standard that you
have a producer
that's going to send out
a message.
All right.
Let me redraw this again really
fast here.
I got messages that I'm going to put
in.
OK.
And so this could be
one way, they have could have one
channel.
All right. So there could be one
way where they want to go out,
someone who consume it.
I could put a request out,
someone gets it, does some
processing.
But the response to that that they
want to do, I'm going to also
be listening on
another queue
for a response to that.
So that could be now that all of
a sudden
where they are reading this request
that comes through that they could
put the resulting workload
or whatever computation that they're
doing, whatever algorithm that must
be done.
Say, for instance, maybe you're
some of those patterns where you
want to put your information in and
look at all the offers that are out
there. It could be a actual search
to go out and get all this
information. Compile that back
to maybe a list of object,
that information comes back and
then I'm listening, the producer is
also listening to another queue
to say, "Oh, the response
is ready." And then
it takes that information and then
presents it. So you have
point-to-point.
All right, this is a Pub.
PubSub and then more of a
request and reply.
All right. So a different type of
messaging pattern that can occur.
And again, I love to kind of think
through things from a backwards
perspective.
It's really about as a technology.
Isn't it a solution architect?
Sometimes you want to put your hat
on where you say,
"What is my intent?
What do I need to happen?" And that
really drives what particular piece
of technology can drive
that particular component here.
All right. So three patterns we
identify message queuing.
Let's wrap it up with the benefits
naturally of why you want to
facilitate or go through the effort
to orchestrate something like this.
And I like to bring this down to
two major
particular pieces here.
The first,
decoupling.
All right, it looks like an N, but
it's really an I in an N, all right.
Trust me there.
All right. So decoupling here.
So what this allows me to do,
we mentioned the first part about
breaking applications apart.
It allows me now to have
separate items that can
occur. So I can,
in going along what we like to call
the microservice pattern,
responsibility pattern, I can
dedicate one particular piece
of code application
to just be responsible
for that one piece of work.
All right.
And if I was in a monolithically
kind of code base, I
would have to progress things all at
one time. If I wanted to just work
on my payments module, I
would also have to test everything
to make sure how that affects other
parts here.
Well, and this particular piece,
let's see, I just wanted the web
front end to be the web front end.
I don't want you to handle a lot of
the processing and doing all
the database connections from there.
I'll let other systems do that part,
or they can be very well at just
connecting to my database,
connecting to my services and my
systems of record, anything in that.
So, decoupling allows you to really
separate things up, where
apps
can just be
apps, you know, just
perform their particular job
here as well.
And naturally,
that will
enforce scalability in your
application, because now
you have apps that are decoupled,
you have teams that can work on
applications, I can think about my
application much simpler.
I may have one component that I want
to work on at a time while the other
is guaranteed to function
as is. And also it
allows me to add more resources to
certain parts.
As we mentioned in the other use
case earlier, where I have a web
form that wants to maybe
handle a lot of high traffic of
submitting on my front end may have
a lot, but I may want to wait
before I process a lot
of that on the backend and not have
that immediate.
I may want to put it through more
checks, maybe more detection,
maybe more actual code
to detect for any of these things
or the use cases are really numerous
here. I'm trying to simplify down
to some something simple that we can
digest here, but
this allows me to have some things
that can scale out to meet demand
without having to
scale the whole application up.
So I immediately get that piece
here.
And so from a perspective of
processing of data,
this is great for me.
And I can also,
evolve other sections of my
code to work better, scale faster.
I want to swap out different
components.
All right.
And even in my decoupling.
Whereas before monolithically,
I probably am writing everything in
one language because it's all one
single code base.
Decoupling gives me the ability to
maybe lean on other languages for
a certain expertise.
I may have my front end in a Java,
but my back end may be in a
Node.js or any other language,
Python, I can take advantage of
many different other benefits of the
language you can offer.
Because again, it's all about the
single responsibility of work that
they must achieve here.
So, to summarize
it all, message queuing, it's
definitely for the new developer.
It is something to consider
from an architectural perspective of
what you want to achieve.
For our existing developers, I
started out as a LAMP and a MEAN
developer myself, started
out for monolithically moved in the
HTTP communication between services.
And this is definitely a technique
that I have embraced in my
hybrid architecture to say there are
some circumstances when these
patterns resonate with me.
I'm like, "yes, that would fit
perfectly to work." Definitely,
if you're getting in the cloud
native being that we have
things that are decoupled
or at the part that you need
many different pieces to function
together to connect them together,
message queuing is a great
segue way to get you immediately
there in the cloud native world.
So definitely a great technique
to try here.
Hope this video was useful.
I can't wait to hear how you are
going to implement message queuing
in your architecture.
Thank you for your time.
If you have questions, please drop
us a line below.
And if you want to see more videos
like this in the future, please
like and subscribe.