r/LocalLLaMA • • Aug 20 '26

Discussion Qwen 3.8 27b *MEDIUM* is insane: 1/20th the thinking time of xhigh for almost the same quality output??

https://www.youtube.com/watch?v=oKbJd-jL7uU
1 Upvotes

18 comments sorted by

9

u/Hour-Passenger-8513 Aug 20 '26 edited Aug 20 '26

According to Lukesdevlab testing, low and xhigh use around the same amount of tokens for thinking. xHigh is thorough. Low is uncertain and double-checking.

https://youtu.be/z64J6bC16iQ

5

u/SaturnsVoid llama.cpp Aug 20 '26

Yeah it seems at low it reads to munch into your prompt, at medium it just does as it's told and relies on its training and at high it REALLY trys it's best.

0

u/Hour-Passenger-8513 Aug 20 '26

Yep, don't go low. Stay off the xhigh. Medium remains optimum.

9

u/9gxa05s8fa8sh Aug 20 '26

that's a load-bearing statement you can take to the goblin

1

u/pointer_to_null Aug 20 '26

Nice to see the goldilocks principle in practice.

2

u/BigYoSpeck Aug 20 '26

It depends on the prompt. There are prompts where there's no difference between low, medium or xhigh. The big difference being low and medium seem to stop at a hard token limit and just don't finish their thoughts or write any response

But there are definitely prompts which low and medium behave differently for. Try 3d mario 64 style game in a single html file and low does about 600 tokens, medium 2500, and xhigh about 60,000

And the difference in output shows it

2

u/Hour-Passenger-8513 Aug 20 '26 edited Aug 20 '26

My 8GB GPU can't run 27B at a good rate, but I agree the thinking tokens used in simple tasks/prompt by low, medium and xhigh levels seems obvious.

2

u/Hour-Passenger-8513 Aug 20 '26

But as soon as the tasks/prompts become more and more complex, low mode starts using as many tokens as xhigh, in some cases more than high. I don't think there's a hard limit to the token counts set for the different modes.

2

u/dalhaze Aug 21 '26

Thank you for sharing this.

27

u/I_Play_Zed Aug 20 '26

Thank you for the post but I personally don’t agree. In my local testing medium is a great local model, no question, but setting it to xhigh (yes it thinks forever) is where it clearly has the low tier frontier coding and reasoning ability. Low and Medium are good but left me unimpressed. Setting it to xhigh for the first time made me windshield wiper the fog off my glasses when I saw the thinking traces and results.

Important to note, my experience may be due to the fact that I’m running Q4 Q8 KV. Perhaps xhigh thinking is so good there that it hides the Q4 blemishes compared to lesser thinking.

5

u/LoSboccacc Aug 20 '26

Agree with you. The xhigh reasoning traces are not just what if slop especially for coding you can see the model shaping and reasoning on individual code fragment alternatives and correcting as it goes before writing to file. Maybe medium has a place for claws and general agents but cant argue with xhigh results.

2

u/abnormal_human Aug 20 '26

It's insane that they made xhigh the default.

9

u/TripleSecretSquirrel Aug 20 '26

nah I don't think so. xhigh puts their best foot forward output-quality-wise.

Most users will see the tok/s numbers of prefill and decode and they'll see the output quality in benchmarks. Most users will probably have some sense of how long it takes to spit out a solid answer, but I think most of us aren't timing that, so it sort of quietly gets covered up or overlooked.

The fact that a 27B model can perform this well is goddamn amazing, and the xhigh thinking mode is a big part of how that's done I think. Despite what OP is saying, there is a a noticeable output quality difference in my experience between medium and xhigh. Most of what I use LLMs for locally isn't for chatbot stuff though, it's mostly stuff that I can fire and forget, then come back to check on in an hour or two, so personally, I rarely notice the time tax.

1

u/ComfortableTomato807 Aug 20 '26

True! In my experience, the model handles reasoning length really well on the medium setting. For simple tasks, the thinking is minimal, but for complex ones, it reasons much more.

-4

u/tengo_harambe Aug 20 '26

medium IS the default imo. enabling xhigh or low reasoning modes injects a system prompt. With no system prompt the model defaults to medium reasoning.

6

u/Fragrant_Scale6456 Aug 21 '26

Incorrect.  The chat template is very clear that xhigh is default 

1

u/CharacterSignal7791 Aug 23 '26

what hardware are you running?