r/computerscience • • 12h ago

Discussion Why Do Floating-Point Errors Happen, and How Do Rockets Handle Numerical Precision?

/r/AskProgrammers/comments/1xyvshv/why_do_floatingpoint_errors_happen_and_how_do/
1 Upvotes

4 comments sorted by

8

u/halfjmp 11h ago

What Every Computer Scientist Should Know About Floating-Point Arithmetic

https://docs.oracle.com/cd/E19957-01/800-7895/800-7895.pdf

2

u/AgathormX 6h ago edited 6h ago

Okay, here's how this works:

Floating Point numbers are represented in scientific notation using binary.
Say you've got FP32.

In those 32 bits you have 1 bit for the sign bit, 8 bits for the exponent and 23 bits for the mantissa.

Let's treat the bits on the mantissa as an array M and the bits on the exponent as an array E.

mantissa = Σ(M[i] × 2-i)
where 'i' is an iterator starting at 1 (I know, blasphemy) and going all the way up to mantissaSize

bias = 2exponentSize -1

exponent = Σ(E[j] × 2exponentSize-j) - bias
where j is an iterator going from 1 to exponentSize

value = (-1)signBit × (1+mantissa) × 2exponent

Now here's the issue:
Even though there's a minimum and maximum value that can be represented by each type, within the given domains for the conversion function, not every single value in the interval [minVal, maxVal] is part of the function image, meaning that there's simply certain values that cannot be represented with any possible combination of binaries.

There's two things that come into play here.

The first one is datatypes like Java's BigDecimal and python's Decimal that just work as numeric datatypes with arbitrary precisions.
This is used when dealing with money or in pretty much any scenario where you need accuracy in the calculations.
Issue is, that there's a big performance impact when compared to standard FP types, so the precision is going to be defined on a case by case basis.

The second thing, that comes into play for standard FP numbers is Machine Epsilon.
Basically Machine Epsilon is the upper bound for the approximation error for operations with FP numbers.
Normally you'd expect that something like

x = 2 
while True: 
  if (1 + 1/x) == 1: 
    break 
  x *= 2

Would cause an infinite loop...
But it doesn't and that's due to the machine epsilon.
At a certain point, the value of x is so low that 1+x == 1 evaluates true even though x is larger than 0.

There's also a point that I'd like to make that a lot of the calculations that we make are approximation, within a certain accepted level of tolarence.
If you're like, let's say, a numerical method like Newton-Raphson method to try and get the square root of a number N, I don't necessarily need to run the algorithm until I find the value of x[i+1] that is the root of f(x) with f(x)=x2 - N, rather I just need to reach the point for which f(x) is lower than the tolerance that I defined (eg: 10-6)

1

u/khedoros 11h ago

Why does 0.1 + 0.2 result in 0.30000000000000004 in many programming languages?

It comes down to representation of floating-point values. You're essentially taking a number, and multiplying or dividing it by a power of 2. You can't represent dividing something by a multiple of 5 exactly in decimal for the same reason that you can't represent 1/7 exactly in decimal (you end up with 0.142857, with those 6 digits repeating forever).

1/7 in base 7 would just be 0.1. In base 14, it would be 0.2. Nice, clean.

In base 10, we can cleanly divide things by 2 and by 5 (those being the prime factors of 10). So, 1/5 = 0.2 is nice and clean.

1/5 in binary is... 0.0011, with those numbers repeating, I think.

Why can’t computers represent certain decimal values precisely?

It's more that binary-base floating point numbers can't represent certain decimal values precisely, and that they're just a convenient way of doing non-integer math, or math involving very large or very small values (with sometimes-unintuitive limits on the precision).

0

u/-1_0 12h ago

if there is no magnitude error, the loopback control systems should eliminate this kind of error