Section 5.2, Archive Shift.
Now here's a graphic depiction of the shifting
algorithm. Basically, it works something like this.
When this archive, the Initial
Primary Archive, which is at Position 0,
when that archive fills entirely,
or I should say fills to
99.8 percent fill, or
if it only has 30 minutes of space
left in it, then, at that point we will do
a shift. As I mentioned, that's user configurable. Mention that
later on, how you can change that, that
fill trigger. But anyway, when that is
just about full,
we will initiate a shift. What we are going to do is we are going to look for the
highest shiftable archive.
Now the highest shiftable archive is either going to be
a, an empty archive
or it's going to be the oldest archive if
there are no empty archives. So, basically, you
can quote this, if it exists, an empty archive
will be selected to be the new Primary. If
no empty archive exists, then the oldest
archive will become the Primary, and its existing data will be
overwritten. So, in this event
here, we have a Position 0 which
was the Primary Archive. We
have filled this entirely, so as of
the time of the next shift, we are going to take this
highest empty archive and we are going to make that
the new Primary Archive. So, after
the first shift this is what you see. This
used to be over here as the highest empty archive. Now it becomes
the Primary Archive. And what
was piarch001,
was the Primary Archive, that's now the
archive in Position 1. So, Position 1 is always the
second most recent archive. Position 0
is the Primary. This is the one most recent
next to the Primary.
Now, as I said, this only happens when the
Primary Archive is just about full.
And, the rule is as I said, we use the
highest number empty archive if an empty archive exists
or if they are all full, the oldest archive is used.
Now, alternately, you actually can do a
force shift. If you look at
the PI System Management Tools, within the
Operations Section within Archives, you can
select piarch001 and -- now here's something
you should not do at home. You can right-click on that and choose
force shift. You can also see a button up here that does
the same thing, Force Shift. So, let me demonstrate.
I am currently looking at Archives. This is Archive 0,
1 and 2.
If we move this back -- I will go ahead and force a shift again.
You should never do this at home unless you are working
with tech support on some kind of a problem. I will go ahead and force
the archive to shift. I will then
re-sequence this. Once this shift is finished, I will re-sequence
this in order 0, 1, 2.
And once we do that, we should see that
the highest empty archive, which is current piarch003,
the highest empty archive is now
going to be the Primary Archive. So let me go ahead and
re-sequence this. 0, 1, 2.
What you see now -- there we go -- piarch003,
which used to be the highest empty archive,
it's now the Primary Archive. And the Start
Time is just a second ago, and the Current Time
is the end time. And here is the archive
that was previous to that, the Primary Archive. It's now
Archive1 in the order.
Let's take a look at how that last,
or how that shift algorithm would work if we had had,
for example, instead of just the three archives,
we started out with a grand total of ten archives
initially. With ten archives initially, let's say we
started that exactly on the 1st of January
of the year. Then, of course, piarch001,
that's actually the first created and the first registered during
the install, that would be the Primary Archive.
And then these other archives, two through
ten, they would actually be empty at that point and
ten would be the most recently registered.
It would also be the last or the
highest empty archive. The
numbering of the archives is determined by the order in which
they were registered. So, since this is the most recent one,
piarch00 -- or 010 is the most recent
one registered, it's going to be the highest empty archive.
So, when this archive fills entirely,
at that point we need to write to a new archive. So, again, we
go to the highest empty archive. That ends up being Archive
10. Now, do you see how in this case it really,
you know, these numbers here, they really do not mean anything. A, very
soon after you start working with the system, you will notice these
numberings are not going to take any kind of a sequence that's
meaningful. So, we suggest you avoid
you know, in, inferring from the, the sequence that that
has something to do with the time order. So, when this archive
is full entirely, at that point
we are going to shift. Now, the highest empty archive is the one
before Archive 10, which is Archive 9. So, we keep
on doing that. Now, there is one restriction, or a few restrictions
when we do this shifting. First of all, the archive has to be
registered. So, it has to be a, an empty archive that's been
registered. It has to be shiftable.
You can actually make an archive non-shiftable. That's something you can do
over in the Graphical User Interface. Again, if I
go here, I can make this non-shiftable
if I prefer not to let this shift on
top.
And, it has to be writable. Again,
you can make something non-writable. And, it also
has to be large enough to handle the current size of the
Point Database. Now, remember I said that 50 percent of the,
of the -- excuse me -- no more
than 50 percent of the size of an archive can be devoted to
Primary Records. And you need a Primary Record for every
single tag in the PI System.
So, if I use my example from before,
if these are 100 megabyte archives, this should not be a
problem for any of these to be shifted into, because
I have only got a ten thousand point system. A hundred megabyte
archive gives me one hundred thousand 1K records.
So, I can actually go all the way up to fifty thousand points.
Now, a very minor digression here.
Not terribly important, but this is the reason
that, the fact that the PI System cannot
shift into an archive that is not big enough to account
for the Point Database. That's a good reason why
we always shift to the highest empty archive.
See, let's just suppose that your system has grown,
and instead of ten thousand tags now it's forty thousand
tags. And then maybe next time you will shift it will be 55
thousand tags. Well, if you remember,
if it's 55 thousand tags, these hundred megabyte archives are not
going to be big enough. So,
if you, in anticipation of this, created three
or four new archives, and made them all
200 megabytes in size, well, those are now going
to be the highest empty archives. So, it makes
logical sense to shift into those. Those are the ones that you have
touched the most recently. So, they are probably going to be the ones
that can accommodate the current, current Point Database count.
Now extending our
timeline here through, through the
shifting into these archives, and then this archive, and then this archive,
at this point right here -- what is this, about mid-April?
At this point in the timeline, we are
now faced with a situation in which this archive is completely
full, we cannot store any more data into that.
And, all the other archives as you see, have already been shifted into.
They all contain data. Well, at this
point, if there are no more
shiftable, and, and empty archives,
then at this point what we do is we go to the
oldest archive in the system that is shiftable,
and that's, in this case it's going to go back to
piarch001. It's going to overwrite that
immediately. It's going to -- when the shift, it's going to,
to zero this out and start writing to this over again.
So what does that mean? Well, it means that this data right
here, it's no longer going to be available. That's just
basically been thrown away, it's been overwritten. And when we shift
into this, we are going to throw away this data. When we shift into
this, we are going to throw away this data. Now it seems rather
extreme but, in fact, the system was designed so that you
could let it go for years and years without touching it, and
we would simply continue to overwrite data
and move forward in time. Now, presumably,
you are going to have backups -- or we recommend you have backups
before this overwriting occurs. So the systems was
designed to use a finite number of archives, and just
simply continue to overwrite them, and if you neglect
the system -- as we have seen customers do sometimes -- and neglect the
system for years on end -- you know, maybe somebody left the job
and it's a while for somebody else to come in and become the manager --
we do not want the system to break down because it runs out of disk
space. And, a system set up like this
will not run out of disk space. It will just simply continue
to overwrite those archives. And, of course, the danger of this --
we certainly want to make sure that,
at the time this happens where we shift on top of an
existing archive, we have already got a good backup of that archive.
So, if you do not want anything
bad to happen, there's a couple of choices for you. First of all.
Just keep adding empty archives. You know, you add them once a month
or so. As, as long as you keep adding empty archives, then
we will never overwrite archives like we have done here in this
example. And, the other thing to do is to just let the system
add archives for you. We have an auto-archiving
system that's available ver... -- starting
with version PI, PI Version 3.4 and later.
And just make sure you have got plenty of disk space because, see,
now that, that nice little design
goal in which you can ignore the system for years and
it continues to run -- well, if you are auto-creating archives
that really does not happen anymore. Now you do have to make sure you have
got enough free disk space. Of course, nowadays that's usually
not much of an issue.
Now, one last thing I should mention about these archives is
if you do want an archive to persist
for a long time, like, for example, this right
here. This archive represents a time when,
oh, I do not know, the, you know, the chlorine gas went
wafting over the local high school, and everybody's still trying to figure
out who to sue, and who to blame, and whatnot. So, this is obviously
something people are going to look at for a long, long time. You do not want
that to be shifted on to. No problem. Just make it
non-shiftable. Then that will stay on
that system. It will -- basically, we will just skip right over
when it's time to shift again. We will skip right over and shift to the
next one. Now we also suggest that you make
non-shiftable anything that's on removable media.
There are folks that like to mount, or register
archives that are on CD drives, network drives,
things like that. So, if you have
got some kind of a archive set up like that, good idea
to make those non-shiftable.