So it's important to understand these two concepts
of latency and throughput. Latency is the elapsed time
from an event. If it takes me one minute
to walk from my office to the classroom, that's
the latency that I'm going to experience for that event
of walking from my office to the classroom. That's
the elapsed time. Throughput is the number of events
that can be executed per unit time. Bandwidth is
a measure of throughput. So once again with this
analogy of walking to the classroom from my office, if
the hallway is wide enough to allow five, ten
of us to walk in parallel side by side to
the classroom, increases the throughput but it does nothing
to the latency. The latency is going to be determined by
how fast I can walk from my office to
the classroom. So, the difference between latency and, and throughput
is very important to understand. In other
words, I can increase the bandwidth and that'll
improve the throughput but it is not going
to do anything to the latency itself. So
in other words, higher bandwidth does not
necessarily imply lower latency. You'll work hard to
lower the latency. RPC is a basis for
client server based distributed systems. And performance of
RPC is crucial, specifically in the context of this lesson. Latency refers to
the time it takes for an application
generated message to reach it's destination. So
for instance, if you're doing an RPC call from a client to the server, then the
RPC call entails sending the argument from the client to the server.
And there is work to be done here, work
to be done in sending the message, work to
be done here before the server can actually execute
the server procedure. So it's the latency that we
are concerned about. And what we will see is
all the software components that comprise the latency for
RPC based communication. And performance of RPC is very
crucial in building client server systems. There are two components
to the latency that is observed for message communication in a distributive
system. The first component is the hardware overhead and the second component is
the software overhead. The hardware overhead is really dependent on how the
network is interfaced to the computer. So, typically in
any computer, what you have is a network controller that interfaces
the network to the CPU. And typically, the
network controller operates by moving the bits of
the message from the system memory of the
node into it's private buffer, which is inside the
network controller. And this part of it, moving
the bits from the memory of the node
into the internal buffer of the network controller
is accomplished using what is called direct memory access.
Meaning the network controller is smart enough to move the
bits directly using the bus that connects the memory to
the network controller without the intervention of the CPU. And
this is what is called dir, direct memory access. And that's
how the bits are moved from the memory of the
system into the buffer of the network controller, and once it
comes here, the network controller can then put the bits
out on the wire, and this is where the bandwidth that
you have connecting your node to the network comes
into play. But there are also other types of
network controllers where the CPU may actually be involved
in the data movement, and in that case, the
CPU does program I/O to move the bits from
the memory into the buffer of the network controller,
from which the network controller will then put it
out on the network. But modern network controllers tend to
be built using DMA technique, meaning that the
network controller, once the CPU tells the network controller
were in memory the messages to be sent
on the wire, network controller does the rest in
terms of moving the bits into it's internal
buffer, and then from the buffer putting it out
onto the network. The software overhead is what the
operating system tax on to the hardware overhead of
moving the bits out onto the network. So
the latency, if you think about the latency
as a whole for doing a network transmission,
there is the software overhead incurred in the
layers of the operating system to make the
message available in the memory of the processor,
ready for transmission. Once it is ready for
transmission, the hardware overhead kicks in, and the
hardware, the network controller in particular, moves the bits
from the memory into it's buffer and then out on
the wire. The focus of course being an operating
system designers work, is to reduce the software overhead and
take what the hardware gives you and think about
how you can reduce the software overhead so that we
can overall reduce the latency involved in transmission. Which
is a sum of the hardware overhead and software overhead.