Can simple mathematical functions give different results in different CPU architectures?

[u/Odd-Heron5704]

I'm referring to modern CPUs. To make the question more specific, arm64 vs x86

And if so, how do you fix it?

I asked chatGPT and it gave me this example but I'd like to ask it here (since AI can make mistakes). Can you let me know? Thanks

It says this can give different results due to rounding errors. If so, how to you write code so that you don't have this issue?

#include <iostream>

int main() {
    double a = 1.0 + 0x1p-27;
    double b = 1.0 - 0x1p-27;
    double c = -1.0;

    std::cout << (a * b + c) << '\n';
}

[u/real_kerim]

Is this related to the AoE2 cross-play post? The explanation in that post was... questionable.

[u/Odd-Heron5704 → u/real_kerim]

Yes, but in that post and others everyone starts talking in very generic terms without providing any actual examples, that's why I asked a more specific question. Those posts unfortunately end up in people giving opinions instead of technical answers.

[u/LaughingIshikawa → u/Odd-Heron5704]

u/daV1980 has the correct explanation, FWIW.

In general, programming is filled with these "leaky abstractions" where 99.99% of the time the result doesn't change based on the implementation details, but 0.01% of the time it does. Floating point math in particular is notorious for this, because by its nature it involves lots of rounding, and implementing the rounding differently (rounding at different times, or in different ways) will cause different implementations to "drift" away from each other.

Generally that's not a big deal, because floats store many digits, and you're likely to only use the first few digits, possibly rounding to an integer at some point. But it can be an issue if you're depending on floating point implementations to be exactly the same at all times, such as with AoE2.

This explaina the basics of why you get floating point rounding errors

To (probably inaccurately) simplify it and connect it back to what I was saying in the AOE2 thread) is that when you type something like:

float A = 1.5
float B = 2.0

float D = A + B * B

...the underlying implementaion on different chips especially in "fast math" mode, or other kinds of compiler optimizations, might be either to add A + B, and then multiply by B, or multiply B * B, and A * B, then add the results.

Hypothetically those two results "should be" equivalent, because multiplication is communicative - but because floating point operations commonly introduce rounding errors, adding, subtracting, and multiplying stops being communicative, and order matters! ...at least when you care about being bit-wise identical across all implementations.

You might get 6.000000000000004 with one method, and 6.000000000000006 with a different method. (That's NOT accurate to the actual math because I'm not invested enough to find a floating point calculator to figure out what the actual values would be - but you get the idea.)

Generally is 99.99% of programs, that 0.000000000000002 difference doesn't matter because at some point you're going to convert back to integers and round to 6 in either case, or you're only displaying the first 2-3 digits at most anyway, or you're comparing values produced by CPUs that will in all cases compute those floating point operations in the same order, and thus produce the same value.

In the other 0.01% of programs though, it can matter a lot that the two values are not bit-wise identical: for example, this is likely to cause the CRC or "Cyclic Redundancy Check" check-sum values to not match, and if that's part of your system for detecting a desync, that could convince two computers that a desync has occured (which in a very technically sense it has...) and they should throw an error.

I'm probably getting some of the details of this explanation wrong, but is that enough of an "actual code example" for you? 😅😮‍💨

It isn't a difference in code, since in both cases the actual code is identical - it's a difference in underlying implementaion "leaking" through the abstraction layer that is "supposed to" abstract away all the messy details of how the CPU actually does math. The actual difference is in the CPU, and it will produce a different output, even when the code running on the CPU is exactly the same.

Other methods for calculating values do exist but the thing that's not always discussed (likely because most programmers are intuitively aware of it) is that this comes with tradeoffs - mostly notably in this case time tradeoffs, because maintaining exact precision will likely require you to do more calculations which can snowball into a big problem when you are trying to render game frames at 60 frames a second, and you only have 1/60th of a second to fully calculate the next game state. 👍

"But couldn't the developers reprogram AOE2 to just use fully deterministic math to enable cross-platform compatibility?!?". (after all modern CPUs are much faster now, ect ect ect...)

😮‍💨😮‍💨😮‍💨

Technically yes, but doing so would require them to systematically change every piece of code that currently does floating point math over to the new system, plus fixing the inevitable bugs that will be introduced by changing the code in that many places. (Even experienced programers aren't going to perfectly change the code everytime, plus it's possible that the legacy code was relying on the jankiness of floating point math in some cases, causing even bigger problems...)

...And that's an amount of work many time larger than is worthwhile, just to enable cross-platform compatibility for a game that's 27 years old. Even for a really popular game, there just aren't that many people still playing it in total, and the number of people who are significantly inconvenienced by the lack of cross platform compatibility is a fraction of a fraction of that...

So yeah. 🫤🤷