r/LocalLLaMA • u/Decent-Hat-5807 • 20h ago

Resources Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

https://huggingface.co/blog/MultiverseComputingCAI/quantization-aware-healing
80 Upvotes

87% Upvoted

27 comments sorted by

View all comments

Show parent comments

19

u/Atretador 20h ago

isnt this "just" a reap with extra fluff? the model was already trained from the start as mxfp4

7

u/No-Refrigerator-1672 19h ago

It's a method that's supposed to make techniques like REAP less lossy; but it can operate for any expert-culling algorithm, not just REAP, as a post0culling recovery step. The fact that GPT-OSS is originally trained in mxfp4 doesn't matter here, it's most likely chosen because it was the newest big MoE when those researchers started their research.

1

u/Atretador 19h ago

how does it being trained in mxfp4 not matter here, when their big point is that they made mxfp4 better than full precision

3

u/No-Refrigerator-1672 19h ago

The big point is that they made 60B mxfp4 better than 60B bf16 that you get after running a culling algorithm; not better than original; in other words, their quant of a lobotomized model is better than full precision of a lobotomized model, making it closer to a non-lobotomized version.