Welcome back to this series on number formats.
This is a general series on how we represent binary numbers in the digital system, how
we visualize them, how arithmetic operations are done, and all the myriad of problems associated
with them that we need to keep in mind when we are dealing with them.
This series is geared towards people interested in general computing, but the examples have
an emphasis in its application in the audio domain.
Let’s get started.
I had previously introduced the concept of the fixed point format in the earlier videos.
Today, we’re going to talk about the motivations for the need of a new number format, the floating
point format.
So what are we trying to solve when we are implementing floating point numbers?
What problems with fixed point numbers should this new representation format address?
If you look back into the previous videos, and count the number of times I said overflow.
That’s a ludicrous number of times to be stating overflow in 40 minutes of content,
and I’m sorry about that.
But it seems like it’s an issue.
I mean, if just adding 2 arbitrary numbers could result in a number, that would never
fit in the space that was allocated for it, and would flow over it, then it’s a problem.
More generally, range is the problem.
The range is limited with fixed point number format, and we need to address that.
The other issue we saw was with accuracy, or rather the lack of accuracy when dealing
with division operations.
Furthermore, the accuracy is significantly lower at the lower end of the range for fixed
point numbers, and isn’t so much of an issue at the higher end of the range.
We need a number format, where the accuracy is consistent across the full range of numbers.
Let’s try to address these two issues.
Range and accuracy.
And we’ll look at range first.
Let’s take a step back and look at how scientists deal with really large and really small numbers.
Take the weight of the earth 5,973,600,000,000,000,000,000,000 kg.
That’s a lot, when represented in decimal.
It’s a lot more when represented in binary.
You need 84 bits to store this.
That’s not the worst part.
The nearest power of 2 register that can store this number is 128 bits in size.
You don’t find 128bit primitive types in computer language, you’ll have to roll with
your own.
But doing that, you’d also have a significant performance hit, since they can’t be held
in a single CPU register, which in a 64bit architecture is 64bits in length.
Also, this value lays out zeros after 5 significant digits.
That doesn’t mean that the mass of the earth is exactly this value, it just means that
our observations and calculations are accurate only to about 5 significant digits, or it
could also mean that we don’t care about accuracy beyond the 5 digits represented there.
I mean, what does a few thousand kilograms matter when compared to the mass of the Earth?
What about really small numbers?
Take this number for instance.
It’s an approximation of a universal constant in theoretical physics called Plank’s length.
Doesn’t matter what it is, but it’s an exceedingly small number.
How would this be represented in binary?
Sort of like this.
I had to actually constrain the final value to 128 bits, because this fraction can’t
exactly be represented in binary, since it results in a non recurring, no terminating
series of numbers, that stretches on forever.
What I’ve represented here is an approximation.
The next video will actually talk about approximations, and how limiting the binary system is when
we want to exactly represent a value.
But for now, it seems quite silly to represent these sort of really small or really large
numbers in fixed point notation.
How do scientists deal with these numbers?
Wel, they use scientific notation!
I’m sure you’ve seen this representation before, and it’s very convenient.
You can represent any arbitrary decimal number in scientific notation with 2 parts: a real
number called the mantissa, multiplied by the base of the number system to the power
of an exponent.
Here, the mantissa would represent the accuracy of the number, the lengthier it is, the more
accurate the entire number is.
And the exponent signifies the scale of the number.
A positive exponent signifies a number bigger than 1, and a negative exponent would represent
a number between 0 and 1.
Now this notation is a lot more expressive.
You can already envision ways of representing these numbers in binary.
You can essentially have 2 fixed point numbers, one for the mantissa, and one for the exponent.
And you can build some rules into the representation system, such that the number is always normalized
- meaning there’ll always be one digit to the left of the radix point, and this digit
shouldn’t be a zero.
If it isn’t normalized, like in this case, the exponent is adjusted such that the normalized
form is achieved.
If you do this, you don’t need to worry about the position of the radix point.
It’s always after one digit.
And another rule is that the base that’s raised to the exponent is always the base
of the number system.
Very good, we only need two identifiable integer numbers.
one for the mantissa and one for the exponent.
But we don’t want 2 separate numbers to keep track of 1 number, so we can merge them
together, so that the m-bits would contain the mantissa, and some e-bits contain the
exponent.
And let’s throw in a sign bit there as well, to keep track of the sign of the number, either
positive or negative.
And here we have the makings of the floating point number format.
The reason it’s called floating point, is because the radix point isn’t fixed to one
position, the exponent would decide where the radix point should be.
This is great, we have the blueprint for a floating point number format, what’s all
the fuss, why is this number hard to work with, seems convenient enough.
Well, I’ve only been representing these numbers in decimal, for convenience.
In digital hardware it needs to be in binary.
This number here, when converted to binary, would look like this.
That’s a bit confusing.
There are only zeros and ones, the base has been changed to 2 because it’s binary, and
exponent is a binary number as well.
Let’s look at this number now, when we convert it to binary, we get this.
This is a lot more confusing and it doesn’t make much sense, the decimal number clearly
has a terminating mantissa, it has a fixed number of digits, why does the binary number
not?
It seems to have a recurring and non-terminating series of numbers.
To understand the problem, we need to take a deeper look at the number line, and realize
that most numbers that we encounter on a daily basis, are just an approximation, and never
exact.