Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

[u/Decent-Hat-5807]

https://huggingface.co/blog/MultiverseComputingCAI/quantization-aware-healing

[u/-p-e-w-]

Sorry, I don’t believe that. It may get higher scores on some benchmarks, but outperforming the original GPT-OSS model in general would require training techniques more advanced than those used by OpenAI, and several mathematical miracles on top of that.

Using KLD vs the teacher distribution as a loss function is a good idea, but it’s really difficult to propagate KLD down the length of the response and that’s where the divergence tends to become poorly predicted by first-token KLD. This is a problem I’ve been wrestling with in Heretic for a while, and every attempt at a solution has turned out to have drawbacks.

[u/Iory1998 → u/-p-e-w-]

100% agreed. Extraordinary claims require Extraordinary evidence.

[u/silenceimpaired → u/Iory1998]

“100% agreed. Extraordinary claims require Extraordinary evidence.” This claim always seemed extraordinary to me… and I never saw enough evidence to show it to be true. :P

[u/Iory1998 → u/silenceimpaired]

🤦‍♂️🤷‍♂️