So let's talk about cache affinity and modern
multicore processors, in modern multicore processors you have
multiple cores on a single processor, and in
addition to the multiple cores that are in a
single processor, the processors the,selves are also hardware
multithreaded. What hardware multithreading means is that. If
a, if a thread that is currently running
on a processor, on a core C1, is experiencing
a long latency operation, for instance it misses in the
cache and therefore has to go out in order to
fetch the contents from memory, that's a long latency operation.
In that case. The hardware may switch to one of the
other threads and run those. So, in other words, it
wants to keep this core busy. There's only one execution
engine in this core, but it has four threads that
it can run on this core. Depending on what these threads
are doing, if they are involved in long latency operations,
meaning they are going out, they're not switched out of
the processor, in terms of operating system scheduling. It is
just that these are the, the threads that have been scheduled
to run on this core and the hardware is switching
among these threads by itself without the intervention of the operating
system. It is automatically switching among these threads depending on
what these threads are doing. If this thread, does the memory
access which is going outside the processor, then the hardware is
going to say, well, you know this guy is going to be. Waiting
for a while, so let me switch to this guy, and
let him do its, do its work because it's possible that
what he needs is available in the cache. And if this
guy also makes a long latency operation, like a memory access,
then the hardware can switch to this guy. And to this
guy. So if all of these guys are waiting on memory,
then of course the core is not able to
going to be, going to be able to do anything useful until
at least one of these memory accesses are complete. So
that's the idea behind. Hardware multithreading. So it is not
very unusual for modern multicore processors to employ hardware
multithreading. So in this example I'm showing you, there are
four cores and in each core I have four hardware
threads. So it is a four way hardware multithreaded core.
And I'm showing you two levels of caches, L1 and L2 cache. L1 cache is specific
to this particular core, C1, shared by these
threads. Similarly, L1 cache here is. Is specific to
this core C2, shared by the threads that are on it. On the other hand this
L2 cache, is common for all the cores.
So [INAUDIBLE] and any, anyone of these L1 caches,
the hope is that we were able to find it
in the L2 cache. If the processor. Has only these
two levels of caches, L1 cache and L2 cache. This
thing in L2 cache is really bad news [LAUGH] because
then, you're going all to, all to the chip. It's
a long latency memory operation. And modern multiprocessors may in
fact even employ even more levels of caching. In addition
to L1 and L2, there may be an L3 cache.
It's normal to have, modern processors having at least
three levels of caches on the chip. And L1
cache associated with core, and a shared L2 cache,
and a shared L3 cache. So that's the structure
that you might see in modern mulitprocessors. So what
we have to think about now is. Eh, thinking
about this cache affinity and the modern multi core
processes and how the operating system should make its
scheduling decisions. So here again, there's a
partnership between the operating system and the hardware.
The hardware is providing these hardware threads inside
each core. And what the operating system is
doing is picking which threads that it has in its pool of vulnerable threads and
map them on to the threads that are
available in the hardware. And clearly, the scheduling
decision, what it tries to do is to make sure
that. Most of the threads that are scheduled a particular core
may find their working set in the L1 cache if possible,
and similarly the threads that are scheduled on this may find
its working set of [UNKNOWN] cache of C2 if possible, and
so on. And also the other thing that the operating system
may try to do is, if you just take the universal,
all the threads That are currently scheduled by the operating system
to run on all these four cores. You want to make
sure that working set of all these threads are likely to
be found in the L2 cache, because if you're missing the
L2 cache bad news because then you're going outside the chip
and that's a very long latency memory operation. And of course
you can extend this idea if, if there is a third
level of cache but to make things concrete let's just stick
to two levels of caches L1 cache and L2 cache and.
The criterion for operating system is to
make sure that the threads that are currently
scheduled on the processors that's available, all the
cores that are available, what it wants to
try to do is make sure that all the threads will find the contents in the
L2 cache. Because nothing in the L2 cache
is going to be elongating the memory operation.