Floating Point Numbers

What's really floating here?

It's called floating point because the point in the number really floats to different spots. This is just scientific notation. There, you can float the point anywhere with exponents.

Decimals vs Floating Points

Decimals is how we write numbers in base 10. It can be both whole and fractional numbers.

Floating point is just the computer's version of scientific notation.

I always assumed floating point numbers were just two integers split by a dot. That's wrong. A floating point value is stored as one value in a floating point register. It uses different binary handling and standards.

Converting floating numbers to binary

The integer and fractional parts are handled differently. They aren't just two integers split by a decimal point.

Floating points aren't accurate

Why not accurate?

The diagram above is only a mental model for why it isn't accurate.

In reality, to convert a decimal to a floating form, the computer does this:

1. if the number is 585.22, it will convert it into regular number as - $\frac{58522}{10^2} \text{ or } \frac{58522}{100}$
2. Then it will convert both numerator and denominator to binary.
3. Perform division on these binary numbers until the quotient is 53 bits.
4. Then converts the answer to the format mentioned below.

Standards for binary representation of floating numbers

All CPU architectures follow one standard, the IEE 754, for floating numbers. It uses scientific notation and normalization. The integer part of the binary is always just 1. The exponent is base 2, since the value is binary.

Ensuring 1 in integer part.

When we convert a decimal to binary, there will be 1 at some location for sure. The normalization process will move the decimal point to left until it reaches the first 1.

Example - If decimal $13.25$ is converted to binary, it becomes $1101.01$. This becomes $1.10101 × 2^3$

What's stored at the end is - sign bit + exponent + mantissa (binary value after the decimal point) only. Here the main assumptions are -

• The integer part is understood to be always 1. Hence, not stored.
• The size of sign, exponent and mantissa bits are fixed.
• The bias added to the exponent is known based on the register size.
• Exponent is for base 2 since it's binary.

exponent can be positive or negative

The exponent itself can be positive or negative depending on how decimal is moved to get just 1 before the decimal point.

Infinities in floating point

There only fixed size floating point registers in CPU and the available bits is split between exponent and mantissa. Longer the mantissa bits, more accurate the number is. Longer the exponent bits, larger the range of numbers that can be represented.

why exponent bits relate to number range?

Normalization needs just a single 1 before the decimal point. If the number is too small or too big, then a larger exponent value is needed.

This is what's exactly meant by range of numbers.

Infinities are special values that are used to represent overflow in floating point. Here overflow is specifically for exponent value. Below is an example for FP32 (32-bit floating point) representation of infinities.

Sign	Exponent	Mantissa	Meaning
0	11111111	00000000000000000000000	+∞
1	11111111	00000000000000000000000	−∞

Adding Bias to Exponent

This standard uses scientific notation for the mantissa. The exponent can be positive or negative. Still, the goal is to keep exponents positive. That makes comparison easier. The exponent size alone shows if a number is larger or smaller.

Meaning of bias in this context

Bias in English means, having an opinion different to truth. That's exactly what's done in IEE755. The actual value of exponent is biased with a fixed value.

For example, a 32-bit float has a bias of 127. The 32-bit float register reserves 8 bits for the exponent. The exponent runs from $2^{-127}$ to $2^{128}$. The exponent bits must hold $-127$ to $128$. We add 127 to each, so only positive numbers are stored.

Floating point in programming languages

When you create floating point numbers in Java, it converts them to IEE 758 format before storing them. This is a hardware need that all programming languages meet.

Programming language specific implementation

In JavaScript, all numbers are represented in IEE754 format. Meaning even for whole numbers, it has only 53 bits available.

FPU in CPU

The FPU is a CPU component that handles the IEE 754 standard. The ALU is only for integers. The FPU does all floating point math.