Subnormal floating point numbers
Something that I've wondered about for a while: Is there a gap between the smallest representable "normal" floating point number and the largest representable subnormal floating point number?
If I understand correctly, the smallest normal floating point number in the IEEE 754 single-precision format is \(1.0 \cdot 2^{-126}\). By the "leading bit convention" this should be represented with a mantissa that is all zeros and an exponent representing \(-126\).
Meanwhile, a subnormal number has an exponent that is all zeros, and this looks like it should represent a factor of \(2^{-127}\). Furthermore, for subnormal numbers there is no implied "leading one bit." Does this mean the largest representable subnormal float is roughly \(0.99999988 \cdot 2^{-127}\)? That would leave a lot of numbers of the form \(0.\mathrm{stuff} \cdot 2^{-126}\) that are just unrepresentable which seems weird.
Fortunately, that's not what how subnormal number's exponent is interpreted! It's true that subnormal numbers are encoded using an exponent that looks like it "should" represent \(-127\). However, under IEEE 754 (as I understand it) they are supposed to behave as if their exponent part is \(-126\). So the largest representable subnormal float is \(0.99999988 \cdot 2^{-126}\) as you might hope.
In particular, this part of the IEEE 754 Wikipedia page and this Stack Overflow answer reassured me. Wikipedia says:
The leading bit convention cannot be used for the subnormal numbers as they have an exponent outside the normal exponent range and scale by the smallest represented exponent as used for the smallest normal numbers.
Which sounded right, but I needed the answer to reassure me that I read it correctly, because that is kind of a confusing sentence. I could fix it, but that would require thinking, and I've done enough of that for now.
Also I stole the decimal representation of the largest subnormal float from the Stack Overflow answer. No way was I going to do the work to figure that out when they had done it for me. :)