So a distributed system is a collection of
Nodes which are interconnected by a Local Area
Network or a Wide Area Network and this
Local Area Network may be implemented using a
twisted pair, coaxial cable and optical fiber And
if it is a Wide Area Network, it
could be implemented using a satellite communication microwave
links and so on. And the media access
protocols that may be available for communication of
these nodes on a Local Area Network or
a Wide Area Network, maybe ATM or Ethernet
and so on and so forth. That's sort of
the picture of What a distributed system is,
number one. Number two, there's no physical memory that
is shared between nodes of the distributed system.
So the only way nodes can communicate with one
another is by sending messages on the local
area network to one another. So there is
no shared memory for communication. Between the nodes
of the distributed system. No physical memory for communication
between the nodes of the distributed system. And
the third property is the fact the even
computation time, that is the time it takes
on a single node to do some meaningful processing,
that computation time is what we are calling
as the event computation time. That is Te.
And a node may also communicate With other
nodes in the system and that's what we're calling
as communication time or the messaging time, TM. And the third property
of the distributed system is that the time for communication between
nodes in the system, TM. Is much more significantly larger
than the event communication. So these are the three
properties which I would like to think of to make
sure that we have a shared understanding of what
we mean by distributed systems. That they are connected by
some sort of local area network or wide area
network. A collection of nodes. And their own physically shared
memory, so the only communication, only way they can communicate
with one another is via messages that are sent between
the nodes using the local area network. And the
third property, is the fact that the message communication
time is significantly larger And even computation time that
happens on a single node. You probably remember a good
friend, Leslie Lamport. I introduced you to him when
we talked about parallel systems, and I said that we
will see him again. In parallel systems, he's the
one who gave us the notion of sequential consistency, and
the same person that we're going to be
talking about in this lecture, Leslie Lamport. And
in particular LAN port has a definition for
a distributed system and the definition of a
distributed system verbatim goes like this. A system
is distributed if the message transmission time, Tm,
is not negligible to the time between events
in a single process. Cause there's a time
between events in a single process, there's a
message transmission time, and so the definition that Lesley
Lamport gives is that a system is distributed
is a message transmission time tm, is not negligible
compared to the time between events in a
single process. What is the implication of this definition?
Interestingly, even a cluster is a distributed system by
this definition. We've been talking about clusters a lot
when we discussed parallel systems and I told you
that clusters are the work horses of data centers
today. Even a cluster is a distributed system by
this definition because processors have become blazingly fast, so
the event computation has shrunk quite a bit. On
the other hand the message communication time is also
becoming better but not as fast as the computation
time that happens on a single processor and therefore even
on a cluster which is all contained in a
single rack in a data center, the message transmission
time is significantly more than The event time. And
so even a cluster is a distributed system by
this definition. The importance of this inequality is in
the design of algorithms that are going to span the
nodes of the network. What we want to make sure,
because the message transmission time is so significantly larger
than the event computation time on a single
node In structuring applications that run on distributed nodes
of a system like this, one has to be
very careful to make sure that the computation time
in the algorithms that you're designing is significantly
more than the communication time. Otherwise, we are not
going to reap the benefits of parallelism. If, most
of the time you're communicating. That's, the reason why,
this definition of distributed system is extremely important.