that brings me to another background
technology that I have to explain to you
and which is called log structured file
system the idea here is that when I make
a change to file Y meaning I either
append to the file or make some
modifications to it what I'm gonna do is
rather than write the file as is I'm
gonna write the change that I made to
the file as a log record so I have a log
record that says what are the changes I
made to this file X similarly I have a
log record of all the changes I made to
this file Y and this is being done in a
data structure which I'll call log
segment and I'll keep this log segment
data structure in memory of course to
make it fast in terms of the file system
operation so with this log segment data
structure what I can do is buffer the
changes to multiple files in one
contiguous log segment data structure so
this log segment data structure I can
write it out as a file and when I write
it out I'm not writing a single file but
I'm actually writing a log segment which
contains all the changes made to
multiple files and because the log
segment is contiguous I can write it
sequentially on the disk and sequential
writes are good in the disk subsystem
and what we want to do is we want to
gather these changes to files that are
happening in my system in the log
segment in memory and every once in a
while
flush the log segment to disk once the
log segment fills up to a certain extent
or periodically and the reason of course
is the fact that if it is in memory you
have to worry about reliability of your
file system if in fact the node crashes
and therefore what we want to do is we
want to either write out these log
segments periodically or when a lot of
file activity is happening and the log
segment fills up very rapidly after it
passes a threshold then you write it out
to the disk so in other words we use a
space metric or a time metric to figure
out when to flush the changes from the
log segment into the
disk and this solves a small red problem
because if wine happens to be a small
file no problem because we're not
writing y as this on to the disk but
what we are writing is this log segment
that contains changes that have been
made to Y in addition to changes that
have been made to a number of other
files and therefore this log segment is
going to be a big file and therefore we
can use the rate technology to stripe
the log segments across multiple disks
and get the benefit of the parallel i/o
that's possible with the rate technology
so this log structured file system
solves the small write problem and in
log structured file system there are
only logs no data files you'll never
write any data files all the things that
you're writing
are these append-only logs to the disk
and when you have a read of a file the
read of a file if it has to go to the
disk and fetch that file and the file
system has to reconstruct the file from
the logs that it have stored on the disk
of course once it comes into the memory
of the server then in the file cache the
file is going to remain as a file but if
at any point the server has to fetch the
file from the disk it's actually
fetching the log segments and then
reconstructing the file from the log
segments that's important which means
that in a log structured file system
there could be latency associated with
reading a file for the first time from
the disk of course once it is read from
the disk and reconstructed it is in
memory in the file cache of the server
then everything is fine but the first
time you have to read it from the disk
it's going to take some time because you
have to read all these log segments and
reconstruct it and that's where parallel
great technology can be very helpful
because you are aggregating all the
bandwidth that's available for reading
the log segments from multiple disks at
the same time and the other thing that
you have to worry about when you have a
log structured file system is that these
logs represent changes that have been
made to the files so for instance I may
have written a particular block of Y and
that may be the
sitting here next time what I'm doing is
perhaps I'm writing the same block of
the file in which case the first write
that I did that is invalid I've got a
new write of the same block so you see
that over time the logs are going to
have lots of Holtz created by
overwriting the same block of a
particular file so in a log structured
file system one of the things that has
to happen is that the logs had to be
cleaned periodically to ensure that the
disk is not cluttered with wasted logs
that have empty holes and then because
of all rights to parts of a file that is
no longer relevant because those parts
of the file have been rewritten
overwritten by subsequent writes to the
same file so logs as I've introduced you
it's similar to the diffs that we have
seen in the DSM system with the multiple
writer protocol that we talked about in
a previous lecture you may also heard
the term journaling file system there is
a difference between log structured file
system and journaling file system
journaling file systems has both log
files as well as data files and what
journaling file system does is it
applies the log files to the data files
and discards the log files the goal is
similar in a journaling file system and
the goal is to solve the small write
problem but in a journaling file system
the logs are there only for a short
duration of time before the logs are
committed to the data files themselves
whereas in a log structured file system
you don't have data files at all all
that you have are log files and reads
have to reconstruct the data from the
log files