Fixed point number and floating point numbers - the 2 fundamental formats for expressing
numbers on a digital system.
They both have their advantages, their areas of usage where they absolutely shine, and
they both fall short or completely fail in other aspects.
There is no one over the other.
As audio and dsp enthusiasts, you'd need to know what where a particular format is preferred
and under what circumstances they fall short.
Why do digital playback hardware prefer fixed point number format.
Why do DAW softwares prefer to work with floating point numbers?
What even is a double precision number?
And what number format do you use when implementing a dsp algorithm.
This video and the next will address these questions and a lot more.
This video in particular will be about fixed point numbers or integer numbers and the next
one will be about floating point numbers.
Both videos will have the same general topics outlined.
We’ll start off with what the format is, what it looks like and how to visualise it.
The range of numbers it can hold, an overview of the arithmetic operations that can be performed
on the, the computational limitations, and the applications that they are good at.
I just want to make a quick note before we start.
This video and the next will get aggressively technical as we proceed through the list of
topics.
It may not be incredibly useful to musicians and producers, so if you feel a bit overwhelmed
by the content, or if you don’t perceive any use for this topic, feel free to skip
this one.
But if you're getting into audio programming or dsp programming, this is meant to serve
as a fundamental cornerstone on how basic arithmetic operations happen in digital systems.
Alright, I’ve already talked about binary numbers and binary representation in computers,
in the video on Bit Depth.
So, just to whiz through, digital systems, like computers, represent and store information
as a series of binary bits.
Each bit has only 2 possible values, 1 or 0, On or Off.
And the number of unique combinations possible with n bits, is 2 to the power of n.
So, 4 bits can store 16 different possible values.
8 bits can store 256 distinct values.
32 bit can store 4.3 billion unique state values.
So you can see an exponential growth on the order of 2.
Take this 8 digit binary number, 0100 0001.
What does it mean?
Without context, it means nothing.
It’s just 1 combination of 8bits.
Meaningless.
You can choose to give it some meaning, based on some rules.
It can be interpreted as the decimal number 65, or it can be interpreted as the English
alphabet A, or it can be a word, like Hat.
As long as your rules apply consistently across a set of binary numbers, your rule can be
whatever you want it to be.
Ofcourse, a rule is quite useless if nobody else follows it.
So there has to be a generally accepted way, to view this binary number such that everyone
or everything that encounters this number interprets it the same way.
How did we interpret the decimal number 65 from it?
The binary system is just another number system.
The digits are ordered from right to left.
Left being the most significant digit and right being the least significant digit.
And each digit has an index starting from an index of 0 on the right, and 1,2,3,4,5,6,7
on the left.
And the value of each digit on its own is just the base of the number system to the
power of the index.
Here it's 2 to the power of the index.
The thing is, we all intuitively know this, but our intuition is only for decimal numbers.
We don't tend to think about it in this calculative sense when we encounter numbers day to day.
But think about it.
Take the number 123.
What's the value of this number, well it's 1 hundred and 23, we don't have to think too
much about it.
But what's the weight of each digit?
The base of the decimal number system is 10.
The indices starting from the right is 0, 1 and 2.
The weight of each digit is 10 to the power of the index.
So that's 3, 20 and 100.
The total value of the number is just a simple addition of these weights 100 + 20 + 3, 123.
These sort of verbose calculations aren't very useful when you're working within the
same number system.
But when you're converting between 2 number systems, it's quite useful.
In our case, except for the 0th digit and the 6th digit, the rest of the digits are
zero.
Anything multiplied by zero is zero, so we can ignore them.
The 0th power of 2 is one, and the 6th power of 2 is 64, so just add all the results together
and you get the decimal number 65.
Just a simple conversion from the binary number system to the decimal system.
So far, we’ve looked at whole numbers or natural numbers, where they’re all positive.
But of course, numbers can be negative, and more often than not, they are in the audio
domain, so we need to be able to represent integer numbers.
A number can be a fractional values as well, so being able to represent rational numbers
is quite useful as well.
So, in digital systems, we have a fixed number of bits that we can store our numbers in.
So, the easiest and most obvious strategy is representing the numbers such that the
difference between the consecutive numbers is always the same.
So this means we put a linear spread of values into the available range.
For example take 8bits of storage.
We can technically represent 2 to the power of 8 or 256 distinct values here.
So there are plenty of ways we can linearly spread the values within this range.
We can go from 0 to 255.
All numbers are positive.
This is also called an unsigned representation, because the sign of the number is pointless
since we’ve agreed to store only positive numbers.
The difference between any 2 consecutive numbers in this range is always 1.
If you want negative numbers as well we can sacrifice 1 bit to store the sign, usually
the left most bit, and we can use the rest of the 7 bits to represent the number.
if the sign bit is 0, the number is positive, if it’s one, then it’s negative.
So you can technically store a range of values from -127 to +127, and again, the difference
between each consecutive number is.
This is called the sign and magnitude representation.
This may look good on paper, and is quite intuitive for us to understand, but is not
efficient when implemented in hardware.
So there’s another representation model called 2’s complement, that’s predominantly
used everywhere.
We’ll look at that soon.
Another representation model is to normalize the range and represent values between +1
and -1.
So all numbers would be fractional in this case.
Going with the same route as above, sacrificing 1 bit for the sign, we can represent a range
of values from 0.9921875 to - 0.9921875, with the constant difference between 2 consecutive
numbers being 0.0078125.
Alright, that’s super unintuitive, how did we get there?
When we determined the weight of each number by considering the index, we assumed an invisible
binary point to the right of the number.
Similar to decimal points, a binary point, or more generally a radix point splits the
number between the whole part and the fractional part.
When no radix point is found, we generally assume that one is present to the right of
the number, or right of the LSB.
But the radix point could be at any arbitrary position.
If the point is over here, we call this the binary 8, 5 representation, where you have
a total of 8 bits, and the radix point is on the fifth position.
In our case, where we were purely representing fractional numbers, the radix point was over
here.
Essentially this last bit was used for maintaining the sign, and if you had wanted to represent
an unsigned fractional number, the radix point would’ve been at the left hand side of these
8 bits, or left of the MSB.
The index of each digit depends on its position with respect to the radix.
So going towards the right side from the radix, the index is from -1 till -7.
The weight of a digit is still 2 to the power of the index.
So when we calculate the weights of the digits, we get the following values.
Fractional parts are particularly hard to get a handle on from the lens a different
number system.
But look at this number in binary, it’s simple and computable for a computer.
In fact the position of the radix makes near to no difference when the producer of the
data and the consumer of the data agree upon the position of the binary point beforehand.
It could be here, here or here.
The range may vary but the arithmetic operations on it stay consistent irrespective of where
the radix is.
More about this later.
But we can generalize a number with a specified radix position as being a fixed point number.
The fixed point here is the binary point, which separates the whole part from the fractional
part.
Generally speaking, the binary point is either to the left of the MSB, where the number is
purely fractional.
This sort of representation is quite commonly used in hardware playback units and rudimentary
DSP converters.
Or the binary point is to the right of the LSB, where number can only be integers.
This representation is probably the most widely used.
In most programming languages like C, C++, Java, fixed point numbers are represented
purely as integers.
And in most operating systems and modern architectures, an integer, or an int represents 4 bytes of
data, or 32 bits of resolution.
A good reason why this representation is used over the other, is that no matter how big
the resolution grows, the difference between 2 consecutive numbers is always the same - its
1.
Fixed point numbers where the binary point isn’t either of the 2 positions, and is
some position in the middle, is rarely ever used in audio.
If you want this sort of behavior, you’ll have to write your own implementation.