So far, we've seen three different memory consistency
models. One is the sequential consistent memory model, the
release consistent memory model. And, strictly speaking, I would
say, the eager version and the lazy version are
just variants of the same memory model, namely
the release consistent memory model. And now we're going to
transition and talk about software distributed shared memory, and
how these memory models come into play in building
software distributed shared memory. So we're dealing
with a computational cluster, that is, in
the cluster, each node of the cluster has its own private physical memory, but
there is no physically shared memory. And
therefore, the system, meaning the system software,
has to implement the consistency model to
the programmer. In a tightly coupled multiprocessor,
coherence is maintained at individual memory access level by the
hardware. Unfortunately, that fine grain of
maintaining coherence at individual memory access level will lead to too
much overhead in a cluster. Why? Because on every load or store instruction that
is happening on any one of these processors, the system software has to butt
in, and implement the coherence action
in software through the entire cluster. And
this is simply infeasible. So what do
we do to implement software distributed shared
memory? So, first part is to implement this sharing and coherence maintenance at
the level of pages. So the granularity of coherence maintenance is at the level
of a page. Now, even in a simple processor or in a true multiprocessor, the
unit of coherence maintenance is not simply a single word that a processor is
doing a load or a store on. Because in order to exploit spatial locality, the
block size used in caches in processors tend to be bigger than the granularity
of memory access that is possible from individual instructions in the processor.
So we're taking this up a level and saying, if you're going to do it all in
software, let's keep the granularity of coherence
maintenance to be an entire page. And
you're going to maintain the coherence of the
distributed shared memory in software by cooperating
with the operating system that is running
on every node of the processor. So what
we're going to do is, we're providing a
global virtual memory abstraction to the application program
running on the cluster. So the application
programmer views the entire cluster as a globally
shared virtual memory. Under the cover, what the
DSM software is doing is, it is partitioning
this global address space into chunks that are
managed individually on the nodes of the different
processors of the cluster. From the application point
of view, what this global virtual memory abstraction
is giving is address equivalence. And that is,
if I access a memory location x in
my program, that means exactly the same thing,
whether I access the memory location x from processor
1, processor 2, and so on and so forth. That's the idea in providing
a global virtual memory abstraction. And the
way the DSM software is going to handle maintenance
of coherence is by having distributed ownership
for the different virtual pages that constitute this
global virtual address space. So you can
think of this global virtual address space as
constituted by several pages, and we're going to
say some number of these pages are owned
by processor 1. Some number of these pages are owned by processor 2. Some number
by processor 3, and so on. So we split the ownership responsibility into
individual processors. Now what that means, is that the owner
particular page is also responsible for keeping complete
coherence information for that particular page
and taking the coherence actions commensurate with
that page. And the local physical memories are available in each one of this
processors is being used for hosting portions of the global virtual memory space
in the individual processors commensurate with the
access pattern that is being displayed by
the application on the different processors. So for
instance, if processor 1 accesses this portion of the
global virtual memory space, then this portion of the
address space is mapped into the local physical memory
of this processor. So that a thread that
is running on this processor can access this portion
of the global address space. And it might be
that same page is being shared with some other
processor n over here. In that case, a copy of this page is existing in both
this processor, as well as this processor. Now it is up to the processor that is
responsible for the ownership of this particular page
to worry about the consistency of this page,
that is now resident in multiple locations. For
instance, if this node, let's say, is the owner
for this page. Then this node will have metadata that indicates that this
particular page is currently shared by both p 1 and p n. So that
is the directory that is associated with the portion of the global virtual
memory space that is being owned and
managed by this particular processor. So statically,
we are making an association between a portion of
the address space and the owner for that portion
of the address space in terms of coherence maintenance
for that portion of the global virtual memory space.