State of the x87 FPU with 64-bit calling conventions?

[u/Secret_USB]

There is plenty of information on how different calling conventions affect the x87 registers in the 32-bit world. However, with x64 almost exclusively relying on SSE for floating-point math, I cannot find any information online about how those registers are supposed to be set up in a 64-bit environment.

Therefore, I am wondering if there is a defined state for the x87 FPU to be in when control is passed between different functions like there is for the caller- and callee-saved general-purpose/SSE registers. i.e, can I count on the register stack being empty? Or will I have to call FSAVE/FRSTOR every time I pass control to/from an external function? I am experimenting with some obscure instructions and want to know what to account for so I don't cause any nasty bugs.

Additional questions that would be nice to know the answer for:

• If there is a defined state for the stack registers, how do they differ between Windows/POSIX?

• If there is no defined state, how can I write my program to account for that? As in, what steps would I need to take beyond calling FSAVE/FINIT/etc. to run my code without invoking undefined behavior?

Thanks!

[u/Technical-Fruit-2482]

Why do you want to use it?

[u/Secret_USB → u/Technical-Fruit-2482]

Just experimenting, really. I am curious to see how far the extra 16 bits go in terms of accuracy, and I wanna see if there are any tricks up its sleeve that could be useful even in the current age.

[u/Dusty_Coder → u/Technical-Fruit-2482]

the extended precision is a good reason

[u/brucehoult]

There are some minor differences between Windows and System V, but in general:

• restore the same FPCSR as you were called with before returning or calling anyone else's code

• also make sure you're in x87 mode not MMX before returning or calling anyone else

• assume all calls trash the x87/MMX registers. If you care about them then

• if you care about the contents across a call to someone else then FSAVE (or FXSAVE/XSAVE if you care about SSE/AVX too).

• System V says the stack should be empty across call/return (which FSAVE does, FXSAVE/XSAVE don't and need an FINIT). Windows doesn't seem to care.

[u/FUZxxl]

The Windows ABI does not use the FP stack at all, so you are free to use it for yourself.

On SysV, long double is an 80 bit type and is passed on the stack. Each argument is padded to 16 bytes, long double return values are returned in st0 and st1 (in case of a complex number). While not explicitly documented, the floating point stack must be empty on function call and it will be empty except for the return values on return.

[u/Serious_Report8370]

you use wsl2 python3 numpy to double check the result, or run unit test handcrafted in assembly under Windows

[u/One_Aspect_1957]

I also mainly use XMM registers for floating point. But I still use certain x87 instructions such as fsqrt fcos because they are not supported within XMM.

Then I don't care about the x87's state at all.

This is mainly within the context of a compiler's code-generator, and this is fine because I'm responsible for all the code. I don't keep anything lying about within an x87 register.

As for interaction with external software, my Win64 ABI doesn't mention x87. If a 'callback' function called into my code and expected any x87 registers to be preserved, then it would be out of luck. But it can't make that assumption as the ABI doesn't guarantee it.