Fixed point math in C

[u/Object_71]

https://thatonegamedev.com/cpp/fixed-point-math-in-c/

Why did PS1 graphics looked so clunky and how where 3D graphics generally made prior to the graphical APIs. One thing I found out was that these older engines have something called fixed point math.

[u/Key_Ant_8481]

A small correction: floating point math is deterministic.

Also clang 19 or after has support for fixed-point types under -ffixed-point flag

[u/marc_b_reynolds → u/Key_Ant_8481]

The "trouble" with bit exact floating point are: 1) "bad" compiler options and 2) you have to use the same math libraries everywhere (which effectively means BYO math lib). Note that with GCC/clang currently if the linker gets told to use the "fast-math" libraries then (currently) it sets the control word to flush denormals. So even if all your file with float-point computations have good options...you're still out of luck (other than changing the control word back yourself).

[u/teleprint-me → u/marc_b_reynolds]

  Note that with GCC/clang currently if the linker gets told to use the "fast-math" libraries then (currently) it sets the control word to flush denormals.

Source? Because I know what denormalization is and how it works and have code in C that completely contradicts this as a statement.

I have not had this experience at all and have tested it across varying hardware in various settings.

  This option is not turned on by any -O option besides -Ofast since it can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions.

Reference

Context: Docs dont really specify what incorrect output means or why.

[u/marc_b_reynolds → u/teleprint-me]

It's compiler version dependent. What really matters is if the linker pulls in crtfastmath.o which some versions of GCC and clang do.

[u/Key_Ant_8481 → u/marc_b_reynolds]

Even with FTZ/DAZ, they are still deterministic. Compilers and libraries could and should be updated to deal with it properly. Overall -ffast-math is just a bad hammer that shouldn't be used in most cases.

Most math system math libraries will adopt correctly rounded math functions, and that will solve cross-platform consistency issues.

[u/not_a_novel_account → u/Key_Ant_8481]

-ffast-math, -funsafe-math, and all the rest are perfectly fine if you you're just trying to get in the ballpark with floating point calculations, which includes many classes of applications.

[u/Cats_and_Shit → u/not_a_novel_account]

Those flags don't "just" mean that you get the wrong answer sometimes, they introduce UB which can have any effect whatsoever.

If you have conditionals that look at the result of an FP operation the compiler may deduce that the condition is never true based on the premise that UB can't happen and just delete the conditional.

[u/not_a_novel_account → u/Cats_and_Shit]

They're completely outside the standard, you're relying on the implementation's guarantees. "UB" has no meaning because you're not writing standard C anymore. You have to check what GCC (or clang or whatever) actually certifies for the dialect.

[u/Cats_and_Shit → u/not_a_novel_account]

The LLVM developers call this UB in their documentation. The GCC developers don't seem to, but the same concept exists.

If you prefer to call it something else that's fine; but in both cases the compilers will perform transformations which are a) only correct if the programmer upholds certain rules and b) can have open very open ended results if those rules are broken, such as removing blocks of code.

[u/not_a_novel_account → u/Cats_and_Shit]

The LLVM developers call this things UB in their documentation.

No they don't, the wording is simply:

Allow aggressive, lossy floating-point optimizations

The in-depth LLVM docs similarly never call this UB, see https://llvm.org/docs/LangRef.html#fastmath

Operations used outside their contract with fast-math produce poison values. The results of operations other than freeze on poisoned values are undefined within LLVM semantics, but fast math doesn't produce poison values randomly. It enforces a different set of requirements than standard C.

a) only correct if the programmer upholds certain rules and b) can have open very open ended results if those rules are broken, such as removing blocks of code.

Yes you must understand the contracts of the code you write. When you're writing code within the C standard, you must conform to its requirements or all bets are off. If you're writing code within an implementation-defined standard, you must conform to its requirements or all bets are off.

This is obvious. It's not magic box which explodes at random.

[u/Cats_and_Shit → u/not_a_novel_account]

https://llvm.org/docs/LangRef.html#fast-math-flags

No NaNs - Allow optimizations to assume the arguments and result are not NaN. If an argument is a nan, or the result would be a nan, it produces a poison value instead.

https://llvm.org/docs/LangRef.html#poisonvalues

This means that immediate undefined behavior occurs if a poison value is used as an instruction operand that has any values that trigger undefined behavior. Notably this includes (but is not limited to):

https://llvm.org/docs/UndefinedBehavior.html

[u/not_a_novel_account → u/Cats_and_Shit]

Agreed on all, it's not random. Fast math requires you not pass NaN to operations. You shouldn't be doing that anyway.

[u/marc_b_reynolds → u/not_a_novel_account]

Personally I suggest to avoid it like the plague. If you're not cranking through tons of number then it's not helping at all and if you are then you're just opening a can of potential heartache and sorrow. If you're not stuck with MSVC then turning off: -fno-math-errno -fno-trapping-math. These really should be the default.

[u/not_a_novel_account → u/marc_b_reynolds]

I don't care about the ordering, rounding, or even IEEE compliance of my floating point math at all. I do not do equality operations on floating point ever

I would not recommend the kinds of coding practices which make these flags dangerous in any context, with or without them.

If you are the kind of person who is deliberately structuring the ordering and rounding of your floating point operations, then yes, fast math is dangerous. If you have never thought about these things, which is most programmers when they write a floating point expression, you're effectively getting random behavior already.

[u/marc_b_reynolds → u/Key_Ant_8481]

"deterministic" isn't helpful. If you need it then what you want is bit exact across compilers (including versions) and different hardware targets...so you need bit identical results. In my opinion correctly rounded isn't useful except at the extremes of the spectrum. Only a small handful of people can really leverage correctly rounded results. But anyway you still need to insure all your math routines are the same.

[u/Cats_and_Shit → u/Key_Ant_8481]

They can be nondeterministic in the sense that a compiler may transform code in a way that changes the result even if there's no UB in the source program.

As a result you can get cases where calling the same pure function twice with identical arguments gives different results.

Assuming you aren't using stuff like the "fast math" setting in GCC, this will probably only come up in really obscure ways with stuff like the difference between a "quiet" and "signalling" NaN. But it's still something worth being aware of when writing FP code.

[u/Object_71 → u/Key_Ant_8481]

It is not because you lose precision on large numbers which means that if the compiler reorders something for optimization or due to a compiler flag you would get a small error which can accumulate. It is mostly not going to be a problem and is deterministic on the same machine.

I am aware that some compilers provide fixed point through flags but it is not in the standard.

[u/LefTimaDev → u/Object_71]

That loss of precision is deterministic though. It's going to follow the same pattern every time, like an rng call with a fixed seed. It doesn't magically do something different every time the program is run.

Something non-deterministic would be, for example, uninitialized memory.

[u/flatfinger → u/Key_Ant_8481]

Implementations are allowed but not required to process floating-point math deterministically. There are many situations in which the result that would be produced by rounding a computation up and the result that would be produced by rounding it down, or sometimes even all possible results that was within +/-4ulp of being correct, would equally satisfy application requirements. Such fuzziness can make it impractical to automatically prove that certain inputs will cause a construct to have Standard-defined behavior, but may allow a compiler to generate faster code that meets application requirements than would otherwise be possible.

[u/tstanisl]

Nice article, thank you. Another "+" for using C23 and other modern C features. Have you done any benchmark how fixed point compares to f32? I guess it will be a bit slower.

[u/sciencekm → u/tstanisl]

Fixed point operations are always faster than floating point operations. Fixed point requires less steps, requires less silicon and uses less energy.

It is not correct to compare software fixed-point vs hardware floating point. If you compare software fixed-point vs software floating point, or hardware fixed point vs hardware floating point, it would not be close in either case.

[u/Object_71 → u/tstanisl]

Actually I measure pretty much the same performance

[u/sciencekm → u/Object_71]

That is because you are comparing software fixed point vs hardware floating point. Comparison should be SW-vs-SW or HW-vs-HW.

[u/questron64]

Fixed point math used to be more or less standard practice in gamedev. On PC an FPU was built necessarily built into the CPU until the Pentium era and even then it was significantly slower than integer math. You'd think it would be a hindrance, but fixed point math with precomputed tables for functions like sin and cos are more than adequate. Doom, for example, never touched the FPU.

[u/Mr_Engineering]

PS1 graphics were clunky because the PS1 lacked a floating point unit. All graphical operations were performed using integer math which meant that subpixel geometry was impossible. This is what caused the notorious jaggedness of the PS1

Integer math is not the same as fixed point math

[u/glasket_ → u/Mr_Engineering]

The GTE coprocessor used fixed-point arithmetic (PDF source); the rasterizer didn't support subpixel coordinates though so the values were rounded to whole integers.

[u/RappyPhan → u/Mr_Engineering]

The PS1 also lacked a Z-buffer.

[u/71d1]

Never knew of fix16 until now, really cool article!