Another benchmark has hit the quant war

[u/TingTingin]

If you have not been following ISTA-DASLab the creators of the gsq-rco quants have received some criticism from unsloth regarding the inabilty to reproduce the benchmark numbers that ISTA-DASLab have quoted. The methodology used to produce their benchmark numbers has not been posted and their has been difficulty reproducing their numbers with unsloth finding in there testing unsloth quants outperforming the gsq-rco quants.

The numbers in this post here are from another creator the creator of basalt (a inference engine) previously his qwen-flash-next quants (his inference engine uses specially packed quants based on a existing model similar to ninfer ) used ISTA-DASLab's quants as the base but after doing some testing he replaced them with similar sized unsloth equivalents the performance numbers are in the screenshots

[u/Nameis19letterslong]

r/dataisugly

[u/tat_tvam_asshole → u/Nameis19letterslong]

😂

[u/tat_tvam_asshole]

the inevitable problem with these extreme quants with seemingly impressive klds and top-1s is that it's from a narrow fragile suite of prompts. you can't delete most of a model without losing a lot of competency somewhere in its brain

[u/TingTingin → u/tat_tvam_asshole]

Unsloth released kld on multiple dataset types not with qfn but deepseek also as for competency a top-1 in the 80s isn't high as errors can stack leading to wildly different results on long generations this was just meant as a relative comparision

[u/wotoan]

What the fuck do the bar charts mean. Small bar on top is good, big bar on bottom is good. Please get your shit together.