For more than 40 years, Landsat satellites have
acquired data from across the globe, with the National
Satellite Land Remote Sensing Data Archive at the
U.S. Geological Survey (USGS) Earth Resources Observation
and Science (EROS) Center in Sioux Falls, SD, holding the
single most geographically and temporally rich
collection of Landsat data in the world.
Over those 40+ years, International Cooperators (ICs)
have played a critical role in
the mission’s success, acquiring Landsat imagery at
numerous International Ground Stations (IGS). Until recently,
most of the data held internationally were unique,
relative to each station's area of coverage, and were
not duplicated in the USGS archive.
In 2010, the Landsat Global Archive Consolidation (LGAC)
initiative began, with a goal to consolidate the Landsat data
archives of all international ground stations, thus making
the data more accessible to the global Landsat community
and significantly increasing the frequency of observations
over a given area of interest to improve scientific uses
such as change detection and analysis. This initiative
is possible only with the tremendous support given by
the Landsat International Cooperators. One critical
component to successfully fulfilling the LGAC initiative
was the ability to recover data from old, obsolete media
for which newer media sources were not available, including
Landsat data contained on thousands of High Density
Digital Tapes, also known as HDDTs. Data recovery from
HDDTs is a painstaking process that is dependent upon
functional aged equipment and diligent operator support.
In an effort to highlight the tremendous work performed by
the data recovery technicians and to shed some light on the
complexities of the data recovery process, this
presentation will summarize the HDDT Landsat data
recovery process. First, boxes and crates
containing HDDTs arrive at EROS from ICs located around
the world. HDDTs are not small or light, with each
tape weighing approximately 14 lbs. The tape contained on
a standard-sized reel is about 2 miles in length and typically
contains one satellite downlink.
HDDTs labelled as Landsat Thematic Mapper (TM) are sorted
out and processed first. HDDTs labelled as Multispectral
Scanner (MSS) or tapes that show signs of mold are set
aside initially. MSS tapes are processed at a later time
by changing a setting in the Ingest software. Moldy tapes,
on the other hand, need to be evaluated, run through cleaners,
and then can usually proceed through the data
recovery process. Once the sorting has been
completed, operators retrieve
about 20 of the TM HDDTs and cart them to the Digital
Camera Lab, which consists of a staging area for the HDDTs,
four ovens for baking, BOW cleaners for cleaning the
tapes, drives used for exercising the tapes, ingest
drives used to read in the data, and a computer station
that displays the data being captured. Some of the
equipment used in the data recovery process was graciously
donated by the ICs and has been vital to the success
of the project. There are four primary steps
in the HDDT data recovery process: baking, cleaning,
exercising, and reading. HDDTs from different countries
require different variations of the data recovery process,
or recipe, and in some cases, multiple recipes for the same
International Ground Station have been necessary.
Identifying the proper recipe can be challenging and
requires patience and persistence from the operators.
The first step in the HDDT data recovery process is baking.
Baking is necessary because of two different but related
scenarios. First, the tape is sticking to the rollers
and guides on the drive, which causes damaged or
broken tape, and second, there is a marginal or bad
quality read of data from the tape.
Each of the four ovens in the Digital Camera Lab
have the same specifications. The ovens are 120 volts,
60 Hz, and 4 Amps. The ovens consist of an enclosed unit
with a door and contain a low level heat source and
small fan for air circulation. The ovens are big enough to
handle 10 HDDTs at a time and have adjustable settings for
temperature and automatic shutoff. The optimal number
of tapes that can be loaded at once varies by International
Ground Station based on trial and error. Once the HDDTs are
loaded in the oven, the door is closed and the oven is
turned on. The HDDTs bake in the oven for 24 hours at
130⁰ Fahrenheit (55⁰ Celsius), after which the operators
either let them cool in the oven or set them on a wire
rack to cool. It is important that the tapes are read soon
after the baking process as the tape tends to quickly
revert back to the condition it was in before the bake.
The second step in the HDDT data recovery process is
cleaning. Cleaning is necessary to remove dirt from the tape
and is performed by using a BOW cleaner. The BOW cleaner
contains a cloth ribbon that runs against the tape to help
remove dirt off of the tape. Operators load the tape onto
the BOW cleaner and run it through twice, with back-to-back
cleanings. The ribbon turns really slowly and lasts about
two weeks before needing to be replaced. If a tape has
not been baked, the ribbon will contain much more black
dirt or soot on it when cleaning.
The third step in the HDDT data recovery process is
exercising. To begin with, operators have to first clean
the drive. They use alcohol, Q-tips, and wipes to clean
the tape path. Exercising the HDDT packs the tape
better on the reel, which then leads to a better read.
The optimal forward and rewind speeds can vary by
International Ground Station. The fourth and final step in
the HDDT data recovery process is reading. Similar to the
exercising step, the tape path is cleaned before
reading a new tape. Next, an HDDT is loaded on the
Ingest drive and the drive is set to run at the speed
at which the data was recorded, also known as the downlink
frequency. The HDDT cannot be read at a faster
or slower speed. As the tape is being read,
there are different things
operators need to watch for, including light activity.
When the Status and Control display shows only a couple
flickers of lights, it is an indication of a good read.
If the Status and Control display shows an abundance
of flickering lights or if solid lights appear, it is
an indication of a bad read. The read head contains 28
grooves and each groove on the head corresponds to a
channel. If a light is flashing for one of the
channels, it is that section of the tape that
the corresponding groove is reading that may have
dirt or soot on it. At the same time, operator
must watch the computer monitors that are
displaying the data capture. As the data is
read off the tape, the image scrolls on the left
monitor along with a data range. The right monitor
displays statistics relating to the image quality.
If the data range rises and the data quality drops,
that is an indication of a bad read. When the read is
not going well, the operator will stop the tape, rewind
it 100 to 200 feet to when the read was going well,
clean the tape path, and then retry reading the tape.
Rewinding the tape and re-cleaning the equipment can
make a huge difference in the read quality of the data.
This can be seen very clearly in the before-and-after image
screenshots. It is rare that tapes from
some International Ground Stations can be read all the
way through with no stops, and some tapes need to be
exercised multiple times to try to repack the tape more
smoothly. Particularly problematic tapes are set aside and returned
to at a later time. These tapes will need to go through the entire
data recovery process again when they are reattempted.
As tapes are completed, they are placed in a finished pile and
later sent to the warehouse for disposal.
Recovering data from HDDTs is a challenging process.
Operators have to make frequent adjustments to recipes,
equipment must be diligently cared for, and multiple read
attempts are frequently required. While the process is labor
intensive and requires tremendous patience and
persistence, success results in the addition of new Landsat
data to the USGS EROS archive, in some cases unlocking new
regions of the world to the user community, and in all cases
increasing the frequency of observations globally.