The directory entry has one dirty bit that indicates that the block is dirty
in some cache in the system, and for every cache in the system, the directory
entry has one bit that indicates whether the block is present in that cache.
If the presence bit is 1 for a particular cache,
that means that we think that cache has a copy of the block.
If the presence bit is 0, that means that we know for
sure that that cache does not have the block in a state that is not I.
So let's say we have one block caches at core 0 and
1, and let's say this is the directory entry for block B,
where this is the dirty bit and these are the presence bits for
cores zero, one, two, three, four, five, six, and seven.
So, this is an eight core system, so
we need eight bits to represent presence of the block in the caches.
Initially, the block is not dirty and is not present in any of the caches.
Let's say that core zero reads from the block B.
The block is not in its cache, so it's treated like it's in the invalid state.
We put the read request out.
Normally, this read request will be put out on the bus.
And because no other cache has this, the memory would respond.
But now we don't have snooping anymore.
So all requests for
a block would be sent first to the home slice of the block.
And we determine what is the home slice by looking at the address of the block.
Different blocks are distributed among different slices so
that we get an even load between the slices.
So let's say we determine that for block B, this is the slice where we go,
and once the request gets there, we look at the directory entry for it.
The directory entry says that the block is not anywhere in the system.
So now we can get the data from memory,
and send the data back to the cache 0, and
we need to tell the cache 0 everything that the bus would normally tell it.
So we tell it that it has exclusive access,
because there are no other sharers.
And we give it the data.
That response gets back to cache 0, and now if it has the exclusive state,
it can put the block in that state and put the data for B here.
When the directory sends the data to Cache 0,
it changes the presence bit for that cache to 1.
Also, we sent the data with exclusive access.
If cache 0 modifies the data, it doesn't have to notify the directory back, so
we also set the dirty bit.
The dirty bit here will not cause a write-back.
It will cause us to find if a cache needs to do a write-back.
So now why does this work better than a bus?
Well because while cache 0 was doing this with this directory slice,
cache 1 could have done something like that for
another block, with another directory slice, completely independently.
However, if cache 1 tries to write to B at about the same time when read
B was occurring, we would send our write request to the same directory entry.
And now, between these two messages,
the directory controller needs to select one.
In this case it shows read request first, so
officially, this read happened before this write.
So the core here gets to read the block B, and
only then, the write gets processed by the directory.
So when this write request arrives here,
again, this write request would be placed on the bus, seen by this cache.
As a result, this cache would invalidate, and
we will get the block in the modified state.
With the directory, we send the request to the directory controller.
It sees that the block is present and possibly dirty in this case, cache 0.
So what now happens is the directory forwards the write
request for B to the caches that are present.
In this case it forwards it to cache 0.
Cache 0 now sees this request just as if it was snooped from the bus, and
because it has the data in the exclusive state it can choose to respond with
the data or it just may keep quiet and confirm the invalidation.
So, what happens here is once the write request has been given to this cache,
we need to send the acknowledgement, at least, if not the data, back
to the directory controller that says that we are done invalidating our copy.
So this point we have invalidated our copy.
When the acknowledgement arrives here,
the directory controller can change this to 0.
If the data didn't arrive, then we can erase the dirty bit.
And now because we don't have the data still, we will read the memory and
send the data to cache 1.
And because we are acknowledging a write request, we again set the dirty bit,
and now the presence factor gets a 1 for the bit that corresponds to cache 1,
and now we get to set M here and write to the B here.
So as you can see, this is how the caches can do their normal coherence,
but now instead of everybody seeing everything,
that we need to send a request to the directory,
which then sends out the messages that need to be sent.
Only to those caches that actually might have the block.
So if a block is shared by only two cores,
then we will be only sending a very few messages.
In contrast, the bus would force us to basically broadcast to all of the cores.
So this also saves a lot of bandwidth because we're using point to point
network links to send just this.
Meanwhile, other cores could be doing something like this with another
directory entry, completely independently over a network.