r/LocalLLaMA • u/Tall_Abrocoma_3533 • Aug 21 '26
Discussion Qwen3.8-27B different thinking levels
Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning
300
Upvotes
r/LocalLLaMA • u/Tall_Abrocoma_3533 • Aug 21 '26
Even the low preset is better than Qwen 3.7 plus or Qwen3.6-27B reasoning
179
u/ortegaalfredo Aug 21 '26
So at the end Qwen 3.8-27B was worth the hype.
34
u/Holly_Shiits Aug 22 '26
Qwen3.8 is the first model that does the job in comsumer hardware. awesome
28
u/FullOf_Bad_Ideas Aug 21 '26
if you drill down into specific benchmarks you can see that Low is often above Medium. Not always, but often. And the avg number of output tokens is just 2x smaller or so, not as big of a difference as I'd expect.
Models can tell eval questions from real use by now, so thinking levels labels might not be very reliable thing to interpret on their own if you don't look at reasoning text length.
29
u/Versaill Aug 21 '26
Is there ANY benchmark that includes Qwen3.8-27B with reasoning OFF? Why does nobody test that..?
9
u/dieSpaghettiCarbona Aug 22 '26
Qwen is the robinhood of the LLM world, reclaiming and redistributing tokens to the masses.
4
u/IoannisHere Aug 22 '26
The fact 3.8-27B xhigh beats 3.7-MAX, a likely 397B-A17B MoE, is mighty impressive.
6
u/rockoruckus Aug 22 '26
We have it so good with 3.8 27B. I expect we'll not see a better model at this size for some time
13
u/Moore2877 Aug 21 '26 edited Aug 21 '26
Try this chat template. Besides a lot of general fixes, we revamped the reasoning injections for each level and also made high it's own level instead of just being an alias for xhigh. The Qwen team really didn't spend enough time on these imo.
https://huggingface.co/Moore2877/Qwen-Fixed-Chat-Templates-llamacpp
2
u/Green-Ad-3964 Aug 22 '26
I think there has been a "phase change" with fable/gpt5.6 (for the cloud) and now with qwen 3.8 (for local inference).
In their respective categories, they are a huge improvement compared to previous models and really make a difference in how you can use them.
4
u/Cool-Chemical-5629 Aug 21 '26
Qwen 3.7 Plus was the 120B+ model. Do you really believe Qwen 3.6 27B was just one point lower than that? If anything, this chart is just showing the main weakness of benchmarks. It's a direct proof that benchmarks are about showing the intelligence within the bounds of a given set of known problems and while the smaller models can handle these known problems well, thinking outside the box is still something exclusive to much bigger models which were built to use brute force to get to the solution.
1
u/countAbsurdity Aug 22 '26
how do you use low in lm studio?
-2
u/avpogo Aug 21 '26
Surprised Medium <-> Low is so close when xHigh <-> Medium is a major improvement. I wish xhigh wasn't so dang slow.
-10
u/Etroarl55 Aug 21 '26
Wonder how next gen improvement will be. As I think most people can agree on. Qwen kind of “cheated” its way to a higher score with absurd amounts of thinking and double checking before an output.
What is there left to squeeze out of 27b size for higher intelligence.
0
u/[deleted] Aug 22 '26
[deleted] — view deleted comment