So far we've been pessimistic that we may have
crashes, but hopefully crashes are not something that happen
very often. [LAUGH] And if crashes don't happen that
often, crash recovery is not that important. But on the
other hand, if crashes are not happening but the
system is progressing along, And what is going to happen
is that we're going to create lots of log records
on the disk as the system is making forward progress.
So we have external data segments, which of course
we need because that is were the persistent objects
are actually contained, but we are also creating these
redo log records. There are reflections of changes that we
made to the in memory versions of these persistent
data structures. Eventually one of these redo logs that
represent changes to the external data segment to be
applied to the external data segments. Now the only time
we're going to do that Is when a crash
happens, that's being very pessimistic. Also, we may end
up clogging the disk with a number of
redo log records. We've seen the need for log
truncation in the distributed shared memory systems as
well. In the case of DSM, those logs were
clogging physical memory. In the case LRVM, these logs
are clogging the disk space. Regardless, these are unnecessary
overhead in terms of space and clutter, and also,
if a particular application needs to map an external
data segment, then we have to know whether that
data segment is up to date or not. And
that depends on whether or not there are some
redo logs pending. To be applied to those external
data segments. So all of these things suggest that
what we need to do is truncate the log periodically.
What exactly is truncating the log? It means that we want to read the logs from
the disk and apply them to the external data segments and get rid of them. Now
this sounds exactly like what I described
to you happens when we do recovery from
a crash. Therefore for Log truncation, simply apply
crash recovery algorithm. So any time the system,
meaning the LRVM run time, decides that it is time
to do some clean up, what it is going to do
is it's going to go and pick some logs to clean,
bring those logs into memory Read the redo log records. Apply
them to the appropriate data segment. And throw away the lock
record. So that's what lock truncation is all about. Of course
we don't want to stop the world in order to do
this lock truncation. So what we want to do we is.
We want to do this lock truncation in
parallel with forward processing by the application. And
the way LRVM allows that to happen. Is it splits the log record into Epochs. It
says this is a portion of the log record that I've chosen the cleanup and this
is a truncation Epoch, and so this is the part that I'm going to use to read
from the desk and apply to the external data segment. And in parallel
with that, I'm going to allow the application to make changes. This is a current
epoch where the application is making changes to the log record.
And the new record which is not yet being used. So, the idea is that
we are allowing RVM to do it's work in terms of Log
truncation by reading a portion of this log
which is a truncation epoch portion of the
log, and applying to the external data segments.
And in parallel with that, we're also allowing
the application to make forward progress by writing
new log records to the current epoch. So
the crash recovery algorithm is being applied to the part of the log that is in
this truncation epoch while allowing forward processing to the
part of the log which is the current epoch
that the server is working on. The biggest challenge
in implementing LRVM is the log truncation code because there's
so much coordination that is needed Between what the
LRV and run time has to do and what
the application may be doing in terms of morphing
the current log segment. You need the log segment for
recovery but it will also overhead when there are
no crashes. And they take up a lot of
disc space. And puts extra burden on mapping and
data segment through the subsystem that wants to use
it. So, managing the logs, truncating the log as
efficiently as possible is one of the heaviest problems
according to the authors of this paper in implementing
LRVM because it directly has a consequence on the performance
of LRVM. And in fact the bulk of the heavy lifting that is done in
implementing LRVM run time and make it really
lightweight and efficient goes in doing this lock
truncation efficiently. What I've described to you here,
is one course level of lot truncation where
We are taking the redo logs and applying
it. A more finer grained way of implementing
log truncation would be to look at in memory copy of the log segment also,
and trying to make sure that we apply it to the external data segments so that
we don't even incur the cost of writing a disk version of the redo log. That
is even more complicated and I welcome you to read details of that in the paper.