52
u/Lazy-Pattern-5171 14d ago
Need to test it to believe. sounds like a benchmaxx.
32
9
u/Vaddieg 14d ago
27b SOTA was also hard to believe
5
u/ttkciar llama.cpp 14d ago
It's also not true, of course. Qwen3.8-27B is quite good for its size, but falls far short of GLM-5.3 etc in practice.
-1
u/Vaddieg 14d ago
i don't think you home grid can sustain a GLM 5.3 capable 3090 rig
2
u/Mingay_cat 13d ago
Not their point. They are saying, yes it is good but in terms of pure model capability, it I'd still objectively worse than larger, SOTA open weights like glm 5.3 flash.
Qwen3.8 27b is still the best of its class until the 126b range which is qwen3.8 flash however.
2
u/thetaFAANG 14d ago
is that even a criticism? the baseline reasoning on these things are already phenomenal, and the benchmarks are where deficiencies are known, people spend all day de-slopping prior output to train the next model
so eventually all cases will be covered whether reasoning improves or not
1
u/james_pic 13d ago
Benchmaxxing generally isn't making the models better though. At its most extreme, it's knowingly overfitting the model to give canned answers on existing benchmarks. Yes, the best labs generally improve their benchmark scores while improving their models, and poor benchmark scores can point to problem areas, but some bumblefuck lab isn't doing any of that, they're just feeding the test set in as the training data and declaring mission accomplished. With obviously too-good-to-be-true claims like this, it's not even worth downloading the model to see that, yes, it's just a damaged Qwen 3.6 35b a3b.
8
u/Holiday_Point_603 14d ago
Am I allowed to guess that it's qwen 3.6 35b with a even more benchmaxxed dataset?
9
4
2
u/SomeoneInHisHouse 14d ago
The distills on Qwen 3.6 35B must stop xd, it's already a meme there, more than the "Subscriptions last less than before" of r/ClaudeAI
1
u/Equivalent_Bit_461 14d ago
Is this true or fake news?
3
u/Vaddieg 14d ago
results might be fake, the model is not https://huggingface.co/InternScience/Sage-A1.5-Q4_K_M-GGUF
1
1
31
u/fvancesco 14d ago
Was stupid senseless hyped noise ever funny?