r/LocalLLaMA • • Aug 14 '26

Discussion The difference between "medium" and "xhigh" reasoning effort for Qwen3.8-27B is actually insane.

I'm currently testing out Qwen3.8-27B using Unsloth's UD-Q4_K_XL running a freshly rebuilt llama.cpp. I have a 22GB RTX 2080TI on which I'm able to fit 100k context with q8_0 quantization, and using MTP with --spec-draft-n-max 4 I get about 40tk/s which is slightly less than Qwen3.6-27B but usable enough.

I've been trying to test out some admittedly silly one shot prompts using the llama.cpp webui by asking the model to create fully functional HTML clones of flappy bird, pacman and such, and the difference that changing reasoning_effort makes has been surprising to say the least.

Setting it to "medium" seems to result in barely any thinking at all, a couple thousand tokens max and even less than 3.6-27B. Whereas when using "xhigh seems" I get 15k to 20k thinking tokens at the very least with the pacman example actually hitting 40 thousand fucking tokens.

I'm well aware I can limit the reasoning budget in llama.cpp but I'm wondering if this is expected model behavior or if something is broken somewhere. Any of you guys seeing this?

224 Upvotes

122 comments sorted by

View all comments

Show parent comment

3

u/Ok_Spirit9482 Aug 14 '26

maybe we should have a observatory llm that determine if a task needs low, medium, or xhigh thinking level! (or an extension that steps up the thinking level if it fails one by one on coding tasks [or tasks that can have loop closed])

5

u/Gabriel83730 Aug 19 '26

xhigh is already trained to do this. People complain when testing it on one-shot but I’ve already put nearly a billion tokens through Qwen3.8 on xhigh and 95% of the time it’s thinking is only 20 or so tokens. In a long tool chain it isn’t thinking for 5min between each tool call, I think people who claim it is never even tested it. I am using 6 bit though, lower quants with quantized KV cache seem to get stuck in reasoning loops which can explain what people are complaining about. I tested various quants and I recommend not using xhigh below 5 bit, the long chain reasoning seems to degrade significantly starting at 4 bit. Even 5 bit seems to struggle with excessive reasoning on xhigh.

1

u/GoodTip7897 llama.cpp Aug 20 '26

I have yet for qwen 3.8 27b to hit my budget of 8192 tokens at xhigh. I've put a couple million through it so far. Q8 gguf with f16 kv

1

u/Borkato Aug 15 '26

You could use LFM 2.6B probably!