Each PI Archive File
consists of a series of 1K records. So these
1 kilobyte records can be divided into what we call
Primary Records and Overflow Records.
And, typically, it's -- I think it's best to think of the Primary Records
being something that gets generated at the very beginning
of your Archive File. So, let's just pretend that this
represents a, an Archive File. You know, this right
here is an Archive File that represents,
let's say, 100 megabytes in size.
What we are going to do is at the
very beginning of this Archive File,
we are going to create a Primary Record for each one
of the tags in the PI System,
that are currently in the PI System. So, for example,
let's say you had a 10,000 tag system.
Well, the 10,000 tag system, if you use
the algorithm I suggested before, 10 - 100K
per tag, that ends up being, at a minimum, a
100 megabyte Archive. So, with a
100 megabyte Archive -- 100 megabytes, that's
a hundred million bytes, that's actually 100,000
1K records., Basically, you are going to have
1K records stretching from,
from Record Number -- let's go with
Record Number 1 over here to
Record Number 100,000 over
here. So, 100,000
1K Records is 100 megabytes.
Well, we are going to occupy the
beginning of this. or the, the
first portion of this in these
Primary Records, one Record per
tag in the System. So, if we have 10,000 tags,
that means the first 10,000 Records
here are going to be Primary Records.
Now, at this point,
some of you may say, well, why in the world do we care about this?
There's actually, there's actually a point at which this may
become an issue. There can be a
maximum of 50 percent of all of
these 1K Records here, a maximum of 50 percent of
them can be used for Primary Records. So that means
that that 50,
50 percent has to be big enough to hold
a, Prim...a Primary Record for each tag in the system.
So that means here, if I were to try to go to.
Let's say 50,000 tags, that's not a problem.
I can, if I look at 50,000 tags,
that would get to just about half
way within these 100,000 Records. If I were to try to go
greater than that, go to, say, 70,000 tags,
at that point you are going to have a bit of a problem
because that's beyond 50 percent of the size of that.
So, if you remembered my
algorithm, I said 10K per tag
or it ends up being about 10 megabytes at a minimum
per 1,000 tags. See, with that algorithm,
you should never run into a problem because with this
10,000-point system, that's a hundred megabytes
by default, we would suggest is the minimum size.
And as you can see, with a 10,000-point system,
it's sized right there to go all the way up to
50,000 tags if for some reason the system
grows larger. So, it's a fairly minor point, but I
do want to point out that the, the PI
Archives do have that one restriction. It's a fairly
minor restriction. Now, let's
talk about what's in that 1K
Record. The 1K Record consists of some Header
information, as you can see right here. 26 bytes of Header information,
which is things like the Point ID and
the Zero and the Span, etc., and the rest of this is
simply where we store the data. So the data's going to be stored
in here in little increments, and the
increments are going to be, be
sized based on the type of tag that it is.
So, for example, if you are working with a
Int16 or a Digital, the data's going to be stored in
3 bytes. If it's a Float32 it will 5 bytes,
a Float64 will be, oops, will be 9 bytes.
And so, using those numbers,
using this 998 bytes of actual storage
space, then
for a typical Float32 we are going to be able to store
about 200 Events in each 1K Record.
So, sometimes people ask, how much data can I store
in a, in my Archives? Well, you now have the ability
to calculate that. Just basically look at what
you are storing, how much space we have per 1K record,
and you can do the calculation.
There are times when you are looking at your Archives,
you will actually see information about things like
Primary Records and Overflow Records, so I would like to explain briefly what they are
and how they fill. When you first create an
archive and register that archive, as I have
described up here, you first create that, that's not
going to -- at this point no Primary Records are created
at that point. You simply have an empty
file with no, no Primaries generated
at this point. Now, when we
shift into that Archive, when the Primary --
excuse me, when the PI Server
has now shifted because the previous Primary
Archive is full, then at that point
we need to initialize this new Archive, and
at that very point, when we initialize that
for the first time, then we create the Primary
Records that go for that Archive, or that are
going to be associated with that Archive. So, if
you have got a 10,000-tag system, then at the point
at which you shift into this previously empty Archive, now
you are going to have 10,000 Primary
Records allocated for that Archive. But they are not
filled yet. Now, as time goes on,
within a few minutes of doing that
allocation, within a few minutes you are going to notice that
those Primary Records are going to start to fill up.
So each one of these Primary Records goes with a different PI
tag. So, let's say that's one tag, Temperature Transmitter,
we will start putting data into that tag. And here's a second tag,
and a third tag, and a fourth tag -- it's not in any particular order.
It's just as the data comes in we are going to start filling
up these Primary Records. So, and as you can
imagine, some of these things are going to fill fairly quickly, and some of
them are going to fill slowly. So, for example, if that's a Lab Input
tag, right there, it may only get one value every day.
So, just gradually as time goes on,
you know, during the day, we are going to be filling these Primary
Records. At some point, we are going to get to a
point -- let's just go forward in time here -- at some point
we will get to a point where one of these PI tags
has completely filled up its Primary Record. So, at
that point there is no more room for data to go
into that Primary Record. So what we do at that point is
we generate what we call an Overflow Record. These
Records that you see over here, these
Records are the Overflow Records. Now,
we do not start by generating as many as you
see right here in red. No. What we end up doing,
we start by generating a single Overflow
Record for that Primary Record that filled.
So, as I said before, let's say this Primary Records fills
completely. At that point in time, we are going to
then generate an Overflow Record. We will create
this Record and then start filling that. And when
that Record fills completely, we are going to generate
another Overflow Record. So, let's say we have
got, we have got these Primary Records that are
all filling, and as these Records
fill, we are going to generate some Overflow Records.
As you can see, they extend from the end,
forward. And then as these Overflow Records
fill -- when -- let's say this Overflow Record fills
right here -- we are going to generate another Overflow Record.
Now it gets actually even more complicated than that.
Once, once we have a PI tag that's
generated more than three Overflow Records,
then we consider that such a busy tag, we start to
treat it a little bit differently. We change this Primary Record
into an Index Record. We basically take all the data
out of it, move that data into a, an Overflow
Record, and instead, the Primary Record then becomes an
Index into where you would find the data in these Overflow Records.
So that's going to continue on and
on, and as you can see, as, as time goes
on, these Overflow Records are going to
eventually fill. And when they finally
fill to a -- this is a
user-definable amount -- when they finally fill
to the point at which we have
very little room left, then at that point we
do an Archive Shift. Now,
we do allow a little bit of space in here, as you can
see, because we do want people to still be able to add data
later on, manual input data that may be out of sequence,
for example. That's basically how the Primary and Overflows
are created. We, first, create
these Primary Records, and we do that at the
time that the -- we first initialize the system, the
Archive. And then, over time, the
Overflow Records are generating from the end of the Record
backwards. So, actually, if you look at the Record
Numbers, the Overflow Records are going to start at the end of
this list, coming backwards. And some of our Utilities
will, will show this.