In this video I'll introduce you
to the archive files by explaining how they work and
interact with the PI System on a high level.
I'll describe the different types of archives and
explain how they are populated with data and tell you how
they work under normal operation and back-filling.
Archive files are a special type of file that are the backbone
of every PI System. These files are
responsible for holding all of your time series data that your PI Data
Archive will collect. PI Data Archives are capable
of holding terabytes worth of time-series data,
representing years or decades worth of data collection.
If you had to store all of your time series data in the same file, it would be bulky
hard to work with and
impact performance. So the PI Data Archive uses
smaller archive files to hold all of your
data. To be efficient, the PI System has to know where
its data is even when spread across multiple archive
files. So to accomplish this, each archive file is
assigned a unique period
of time. Each archive is designated
a start time and an end time. In a
healthy PI Data Archive, one archive file's
end time will correspond to the next archive's start
time. Here we can see that archive 1 starts on
the 13th and ends on the 17th where
archive 2 immediately picks up and runs until archive
3 starts on the 19th.
Cataloged in this way, the PI Data Archive will know exactly
what archive file to look through
users ask for time-series
data. There is always one archive file that doesn't
have a set end time. This is to accommodate all of the
new real-time data that the PI Data
Archive is receiving from different data sources on the network. And it's called the
primary archive. Here, all of my data sources are writing
current, real-time data to the primary archive. And this
primary archive
will receive data until it's holding a
certain amount of data known as the fixed size
limit. Because of this these archives are known as
fixed archives. Once it fills up,
the data will have no place to go. So the PI Data Archive can
automatically perform an archive shift. When
an Archive shift occurs, the current primary start
and end time and is no longer
primary. Instead a new fixed primary archive will be
created and data will then flow into this
new primary archive. This process
will repeat every time the primary archive fills up
to accommodate more data. Sometimes instead of
real-time data, you'll want to write historical
data to your PI Data Archive. This is called
back-filling. Since your archives are
expected to be fixed type archives that result from the
archive shift procedure, normally your archive files wouldn't be
able to hold this back-filled data.
Fortunately the PI Data Archive can automatically
changed fixed type archives to dynamic archives
when they overflow. These archives will have the same start and end time,
but now they can dynamically grow to accommodate new data
as it enters them. The reason we typically only use dynamic archives
for back-filling is that
they are ill suited to be primary
archives due to their constant size changes. The last thing
to know about the archive files is how the PI System
allows users to annotate their data. If
you have a piece of data that could use a comment or some words to
explain an anomaly, the PI System will
let you annotate that event with a manually inputted comment.
PI users will often use these to mark or flag
errors in their data without deleting their
data outright. These comments or annotations
aren't stored in the archive files we've been
talking about. But instead each archive file will have a
small partnering annotation file that corresponds
to it and will hold any comments for the archive's time
period. Annotations are stored in this way
because our archive files are specifically optimized to only
handle time-series data. I hope you are more
acquainted with archive files than you were at the start of
the video. Archive file are critically important to the health with the
PI System and without them you wouldn't have any time-series
data at all. In the next few videos, I'll be walking through some common
tasks to ensure the health of your PI Archive
files starting with a walk through on how to ensure that
automatic archive creation and archive shifts are configured to
OSIsoft best practices.