Remember back in the DOS days when we used to Low Level Format our hard drives, just
because we could?
Oh, the fun we had.
Except, we weren't really low level formatting anything, because low level formatting your
hard drive will render it useless.
Let me explain.
It would be nice if setting up a hard disk was as simple as FORMAT C: /S
But, in the early days of IBM PC Compatibles, things weren't quite so simple.
As well as high level formatting for the OS and partition creation, we needed Low level
formatting, otherwise known as Physical formatting.
Now, the term "Low level format" has been whispered from ear to ear for many a decade,
and what we may call a low level format today, is quite different from what it was.
Since the days of IDE, we can partition a disk, create new file systems and even do
a "Zero-fill" where the disk contents is overwritten with zeros.
This can help weed out bad sectors and allow your hard drive to reallocate them from a
reserve stash, it also renders any previous data un-recoverable, but not so much, low
level formatting.
Indeed the full format we ask DOS or Windows to perform simply writes the file structure
to disk - the File Allocation Table for example - and checks the drive for bad sectors, although
since Windows Vista, this will also zero fill the disk.
A quick format simply blanks the index tables of the applicable file structure, and declares
the job done.
So in reality your data is all still present, but the operating system just ignores it.
But wind the clock back to the IBM XT systems, and we encounter a different ball game.
When first manufactured a drive only has cylinders and often tracks - no sectors.
So then the original meaning of "low level formatting" was a process whereby the sectors
are "drawn" onto each track using magnetism as the "ink".
Each sector was then filled with the lowercase sigma character and the interleave factor
is set.
This is essentially arranging the sectors in the most efficient configuration possible
based on the time between sequential data reads and where the head is on the disk when
it looks for the next piece of data.
This physical format process matches up a drive and it's controller.
As every controller has it's own quirks, a controller may not be able to read a disk
formatted by a different controller, and when the first 5.25 inch hard drive came out in
1980 it required a separate controller card.
Hence a low level format was a good idea to ensure the two were getting along smoothly.
This process could usually be completed using the Advanced Diagnostics disks on XT or AT
machines, and allowed you to set a user defined hard type type with the settings usually stored
in the drive's master boot record.
The process would typically ask you for the number of cylinders, head, the cylinder to
start reduced write current, the cylinder to start write pre-compensation, interleave
factor and the maximum number of error bits to correct.
It wasn't for the faint hearted.
You can also use the DOS debug command to trigger the low level format from the controller
BIOS or even use programs such as HDTEST or Gibson Research's SpinRite, which it self
contained the standard information for lots of hard drive models.
Early hard drives also sometimes had a few unreliable sectors from the go, which would
be noted on a sticker of the drive - these were of course, the quaint, hobby like days
of PC manufacture.
But this information was useful for the low level format, so they could be marked appropriately
as unusable, in case they weren't automatically picked up from the format.
As a loose example, entering the wrong settings on a low level format could mean for example
that a 10MB drive is formatted as a 20MB drive.
This might not be disastrous, but would perform slowly and perhaps be returned as a 20MB drive
with 10MB of bad sectors in diagnostic utilities.
Once all the settings are ready, the drive heads could then perform the low level format,
restoring the magnetic image of the sector IDs as it went.
These sector IDs separate one sector from the next, and especially with older disks,
it was possible for them to fade.
This is typically what could lead to the dreaded "Sector Not Found" errors in DOS.
The data might be fine, it's just that the drive heads can no longer sense the sector
boundaries.
So, you might wonder why we don't perform these low level formats with our current hard
drives, or even hard drives from the IDE period onwards...
Well, there is a good reason for that.
The first is because these disks are "mastered" if you will, using a high enough level of
magnetic energy, that the sector IDs stay intact for a longer period of time.
15 years at the least.
The second was, in the earlier days at least, due to cost & manufacturing.
Take the ST506 drive.
Holding 5MB of data when formatted with Modified Frequency Modulation encoding, it packed about
800 cylinders on a 5.25" platter.
Fast forward 10 years and we had 3.5 inch IDE drives packing 3,600 cylinders.
But with advancement comes cost.
The ST506 drive head had to be precise enough to low level format the drive accurately and
precisely, placing sector IDs exactly, however it was possible to build a cheaper mechanism
that can simply follow sector IDs without having to place them, and this is essentially
how these new fangled drives were built.
They were also mostly formatted with Run Length Limited encoding, which could store more,
but required higher precision when laying out the drive's sectors.
These drives were therefore physically formatted in the factory and had their own on board
controllers, hence eliminating the variance between the controller and the disk layout.
So when, in 1986, IDE (Integrated Drive Electronics) drives appeared, designed to be compatible
with older drives, and appearing the same to the operating system, people attempted
to low level format them, to keep them fully optimised and in check.
Unfortunately, this resulted in a horrific mess and lots of inoperable drives.
Drives then evolved to cope with this new human problem, with some just returning simple
"OK" messages to the low level format request, but actually doing nothing, and others going
a step further with a low level format actually erasing the data, a bit like the zeroing I
mentioned earlier, but leaving the sectors unaffected.
So, then, this is why you should never low level, or physically format your hard drive.
Because frankly, rather than maintaining it, it would ruin it.
Thankfully, since the 90s, drives won't allow you to low level format them, so really you've
got nothing to worry about.
But I still consider it a fascinating view back into the past, exploring the evolution
of our technology.
If you're not interested in that, then I guess well done for watching the video anyway.
Go and make yourself a tea.