Let's now discuss the components of the RPC latency.
By now we are all familiar with the semantics
of RPC, namely, in RPC the client, is making
a remote procedure call to a server, and it
has to send the arguments of the call to
the server so that the server can execute the
server procedure and return the results back to the
client. So if you look at the components of the
RPC latency, it starts with the client, with a
client call. So the client call subsumes a number
of things. Number one, it is setting up the
arguments for the call. The client has to set up
the arguments for the procedure call. And then it
makes a call into the kernel. And once the
kernel is being informed that it wants to make
this call, the kernel validates the call, and then marshals
the arguments into a network packet, and sets
up the controller to actually do the network
transmission. That entire set of activities that the
client program and the kernel is involved in, in
getting ready a network packet to send out,
is subsumed in this one line, which I say
is the client call. The second part of
the latency is the controller latency, and this is
the part where the controller says, well, there is
a message to be sent out. I know where
it is in memory. I have to first DMA
that message into my buffer and then put the
message out on the wire. That's the controller latency
and so this part of it is in hardware,
and as operating system designer we're going to take what
the hardware gives us. Controller latency is what you have,
that given by the hardware. The third part of the
latency is the time on the wire. Now this really
depends, as one might imagine, on the distance between the
client and the server. The limiting factor of course is
speed of light. So, depending on the bandwidth that's available
between the source and the destination, perhaps if you have
to go through intermediate routers and so on. It is
going to take a certain amount of time to go from
the client to the server machine, and that we
call as the time on the wire. So then, the
message arrives over on the destination node, and it
arrives in the form of an interrupt to the operating
system. So, the interrupt has to be handled by
the operating system, and part of handling the interrupt is
moving the bits that come in on the wire
into the controller buffer. And from the controller buffer into
the memory of the node. So all of
that activity is subsumed in this item number four,
which I call the interrupt handling. So once the
interrupt handling is complete, then we can set up
the server procedure to execute the original call. Now,
what is involved in that? Well, you have to
locate the server procedure, and once you locate the
server procedure, you have to dispatch the server procedure.
And once you dispatch the server procedure, you
have to unmarshal the network packet that comes in
as the actual arguments for the call that the
server procedure has to execute. So all of that
setup is first done, and then the server procedure
can actually execute the call. So this is the
five-step process. From the time the client says, I
want to make a RPC call to the point
of actually executing the call, these are the layers
of software and, of course, hardware and time on
the wire, by which time you're ready to execute
the server procedure. So even though it looks like a
simple procedure call from the clients point of view,
there is all this latency to be incurred in
executing a remote procedure call. So at the end
of step five, the server is all set to execute
the procedure. Let's see what happens then. So step
number six is server execution, meaning that it is actually
executing the procedure. And of course, this is not under
our control as operating system designer, because at this point,
the amount of time that the server is going
to execute this procedure depends really on the logic of
the program that has been written as a client server
program. And then, finally, once the server procedure has completed
execution, then it says okay, I am ready to
send the reply back to the client, and that's
where we pick up again, so, what happens is
that at that point, you are receiving the results. So,
once again, just like when the client wanted to
send the arguments, you have to convert the actual
arguments into a network packet and send it out
on the wire. Similarly,when the server is ready to reply,
you have to take that reply, which is the results
of the execution of this procedure, and make it into a
network packet. And at this point, once it has been made
into a network packet it has to be handed over to
the controller, and the controller does exactly what we did
on this side, which is to say the controller latency is
gon, is going to be incurred. So that's why you see
item number two appearing all over again in the return path.
Similar to sending the arguments over to the server
on the wire, the results have to be sent on
the wire back to the client. And so you
see that item number three, which is the time on
the wire, is reappearing on the return path as
well. Come over to this side. The incoming result message
is going to result in a, an interrupt on the
receiving node, the client node. And that is exactly similar
to what happened on the server side item number four.
So you see number item number four reappearing on the
return path as well. So that is the interrupt handling
part. And once that interrupt is handled, the operating system on
the client side said, oh this was on behalf of
this client, let me redispatch the client, set up the
client so that the client can then receive the results,
and restart execution where it left off. So the only two
new things that we added in the return path
was item number six and seven. Two, three, and
four was exactly the same as what we saw
on the way over to the server, that is being
repeated on the way back to the client. So
that's the seven step latency involved in the RPC, not
worrying about the actual execution time of the server
core itself because that is not in the purview of
the operating system, it is in the purview of
the client server program that the app developer has done.