hello friends, welcome to reach goals
today I am going to talk about a system design and this we are going to discuss
about what is message queue and how it is used in the real world so here are
the key discussion points in the slide slide so what is messaging queue a
popular messaging queue software how it is used in the real world and some of
the production issues and if you need more info please comment and subscribe
to get in future videos so what is messaging queue so they are nothing but
a software components used to interact between the two different applications
right so let's say you have an application a and you have an
application B now if you want to pass the data between the application a and
application B then we can use the messaging Q so messaging cube by default
they are in a synchronous in nature so it adds it adds a lot of advantage for
the applications right so what is the synchronous it means the sender and
receiver need not interact with message at seams at the same time right so there
also they are used to decouple heavyweight processing and eliminate
these spikes so how what does it mean is you have an application a and you have
an application B let's say it when there is a point-to-point connection
the sender will send a request to the application B and application B has will
be doing a heavyweight processing and Center B has to wait for what it has
been processor are or how long it has been taking to process right in this way
what happens is there is a heavy transaction happening between the
application and epic application a and application B and it can lead into the
heavy processing and it can lead into system spike right on you need to
increase the memory the capacity etcetera right so in order to avoid
those kind of things you can use messaging queue let's see how it can be
used in the future slides so here you see
popular messaging cue software's rabbitmq Apache Kafka Amazon SOS TIBCO
Enterprise ActiveMQ and these are some of the some of the messaging to you
applications or software's used in the recent recent recent past right I want
to discuss about to real-time use cases about messaging queue and its
applications right so if you look under this screen on the left side you see a
mainframe system which is a legacy one and where you see lot of computers are
connecting to the mainframe applications and all the data gets passed into the
database right this is a typical application if you see in any legacy
applications in the stores you would see this right now let's see there is a
requirement for from the business to generate a report out of this data right
so if you are a modern software engineer what it obviously will think is you will
try to expose a versatile web services from here and and the all these
applications are up app server can interact with the restful web services
to get this information and to create a report right that is modern right so if
you think about the legacy systems we cannot do this because mainframe system
is located in different store locations or in the retail stores and it is not
easy to build this application again or it is not easy to maintain this
application right so on the requirement keeps on changing and you cannot go to
the each and every store and keep on changing right so what usually what
usually the architects do is they will try to expose this data and they will
send it to your messaging queue right so messaging Q is nothing but it's it's a
software application sitting in between and trying to receive all the
informations from the mainframe system in a queue for example you can see the
first one right o ID 1 2 3 / SKU 4 5 6 blah blah whatever it is it could be a 1
queue or it could be a 1 message in a queue and the second one can be the
other messaging queue so you will see lot of messages coming as a queue to
this queue right so now there is a receiving end in the receiving end you
will have a queue processor what it what it does is it takes the messages it
processes it and it puts it to the database right so now there is an app
server as usual and it can process this data
and it show up in a cold report that is one of the use case right so now let's
say there is other requirement coming after a few years right I wants to
generate another report so now you don't want to go into the mainframe system and
do it again you already have a lot of informations in the message and you can
use that messages to build a different queue or to send it a different cue and
you can use that you you can use the same cue processor or a different cue
processor to process this information and create a different report so that is
one one of the real real-world use case which I have seen in the past the second
use case I have is you have all observed that you there will be a lot of access
point on the wireless router within your company right so then you connect
wireless when you connect your phone or a desktop to the wireless what happens
is it gets directly connected to the access point or the routers and lot of
logs are generated from the access point for example it knows who or the user is
like you know the IP address from which location is accessing and many more
informations are tracked in the access point all this informations are stored
in the logs right so now if I want to generate a cool report out of this for
example I want to know how many users are accessing this Access Point or how
many how many users are accessing from which location so if I want to get all
this informations I can get from the logs right so what it does is this law
the log information is passed into the queue like you know each each line could
be a queue and you can have a software like distributed distributed messaging
queue like Apache Kafka and the receiving end is at the receiving end it
starts so receive the line-by-line information now there is a queue
processor like similarly what we have seen the previous use case the queue
processor processes this information and puts into the database now there is an
app server you can write a query to generate that if different reports and
you can show in the dashboard for example as we talked earlier if I want
to get an information how many users are accessing the access point one you can
get that information right so this is the other use case I have seen that I
have seen in the previous projects right
now let's move into the production issue this is the quite often I've seen the
production issue and when we deploy a message queue this message queue
overflow what is Q over flow let's see you have a message you are sending into
the queue and we are waiting to receive and the receiving in it says somebody is
not receiving at the receiving end and you see lot of messages or the messages
gets accumulated thank you what happens is it gets it a spike or it gets
overloaded and CPU can spike or the system can run out of the memory and it
can lead to the crash right how do you avoid this kind of situation what
normally the what normally the support team does is they will have an alerting
mechanism when the queue goes beyond a limit or when the messages in the queue
goes beyond a limit send us an alert so it sends an alert let's say if it goes
beyond 70 percent of its capacity since another so immediately the support
person pitches into that and he checks how it can be eliminated right so
multiple ways they can fix it one is they will see why it is not receiving
they're receiving in and they have they can provide a solution to fix that or
the other option is they can write this messages into a different local store or
different devices and later they can pass into the queue and it can project
start processing when everything is fixed so there are a lot of
configuration or the configuration mechanisms to handle this right this is
one of the issue which you would be observing or noticing if you are having
a message queue in your architecture or in your application right okay that's it
I have thank you very much if you have any comments you can put in the comment
section I am very happy to answer that please make sure are subscribing liking
and sharing so that others also can get this information Thanks