I mentioned earlier it's a multiple writer protocol,
which means that this action that's going on
can be happening simultaneously for the same page
X on different nodes of the cluster. That's
perfectly fine so far as the protocol is
concerned, because the assumption is that the user
has an association between locks. And the data
structures governed by the lock. So, even if
the same page is being modified, hopefully
different portions of the same page has
been modified because concurrently if a page
is being modified, that means that different locks
are protecting the portions of the page
that are being modified by the different processes.
Now if writes are happening to the same portion of a page under different locks,
that's a user's problem. That's a data race. That's
not the problem of the DSM software. It's an
application problem because it represents a data race that
should not have been there if the application is
constructed correctly. But if the application is constructed correctly
and the multiple data structures are hosted in the
same page and the data structures are all governed
by different locks. DSM software has a way of ensuring
that changes made to a critical section under
a particular lock is propagated from one processor
to the next processor where the first processor
is the current owner of the lock, and the
next processor is the next user of the
lock. So this implementation that I've detailed here
is an example of the cooperation between the
distributed shared memory software and the operating system to
make it all happen. And in particular,
TreadMarks implemented this LRC multiple writer coherence
protocol on a Unix system. And in
the Unix system, the operating system generates
an exception called SIGSEGV in the operating system layer when a shared page is
accessed by a thread. This exception is
caught by the thread block's runtime handler.
And at that point the DSM software get
into gear, contacts the owner of the page, checks
the status of the page. And if the page is invalid then it gets the page and
the difts for that page and once it brings in the contents of the page and the
difts it creates a current version of the page.
And if the process that is trying to access
the page is making a read access, then there is
no problem. But if the process that wants to use
that page wants to write to it, at that point
it creates a twin and does all the things that
I just mentioned. So one thing that you will notice
is that there is space overhead for the creation of
the twin and the point of the write. You have
to create a twin. And then at the point of release,
you have to create, of course you can get rid of the twin, but you're creating
a dift data structure. So, the twin and
the difts are all data structures. Of the implementation
of distriuted shared memory. And as time goes
by, there could be a lot of these difts
that are lying around in different nodes. Imagine
that a page was touched by ten different processors.
In that case, there are going to be difts lying
around in ten different processors and if eleven processor wants
to access the same page the DSM software has
to go and bring the dift's from this ten prior
users of the page. Get the original page from
the owner. Apply the difts to create the new page.
A lot of latency in, is involved before the
guy who needs the page now can start using it.
And also there is a lot of space over here in the fact that all these discs are
lying around. So one of the things that happens
in, in the DSM software Is garbage collection. And that
is, you keep a watermark of what is the
amount of difts that have been created in the
entire system. If it exceeds a threshold then you
start applying these dift's to the original copy of the
page, at the owner, so that you can then get
rid of the dift's completely. Why do you need that,
well the difts that going to be lying it on till
a long time till the next time the pages access
by someone, you don't want that. So, what you're trying
to do is, you're reducing the space overhead in the
DSM implementation by periodically doing this garbage collection. And applying
the difts to the original copy of the page so
that he can get rid of it from the system. You don't
want to do it too eagerly, but you don't want to wait
too long also because if a page hasn't been accessed for a
long time, difts are going to be lying around for a long
time. So there will be a demon process in every node that
every once in a while wakes up and sees how much difts
have been created in my known. If it exceeds the threshold then
it says okay time to get to work. Let me go and
apply this difts to the original copy of the
page so that I can get rid of the difts.