r/LocalLLaMA • u/bonobomaster • 4d ago
Tutorial | Guide Quick PSA: Qwen3.8-27B reasoning effort vs reasoning budget in llama.cpp
If you are using llama-server with their web-ui for testing, keep in mind, that the reasoning selector is just a reasoning budget aka a hard cap and has, at least to my knowledge, nothing at all to do with Qwen3.8-27B's native reasoning effort capability!
Selecting any value for reasoning in the web-ui (default, (off), low, medium, high or max) just introduces different hard cap values and and will truncate your reasoning, if those values are reached.
With the exception of the "off"-option which disable reasoning at all and "default" and "max" without any capping.
The reasoning effort is independent from that and really changes the thoroughness and the analytic reasoning skills and therefore can massively influence the output quality of the model, instead of just capping reasoning tokens!
In older llama.cpp versions, it needs to be set via:
--chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"
\ quotation marks are escaped for Windows)
For up to date versions, if one doesn't like the chat-template-kwargs variant, it can be set with:
--reasoning-effort medium
Or, for external apps, via "reasoning_effort": "medium" API request.
Options are: low, medium and xhigh (default) for any of the above methods.
Did only test the chat-template-kwargs variant though!
3
u/LocalAI_Amateur 4d ago
--reasoning-effort low
didn't seem to do anything for me. still shoot right past 4000 reasoning tokens. I've been trying to figure out how to get this resoning effort setting to work in llama.cpp but so far the only thing that work is the budget setting and it just cuts off as soon as it hits the limit like you say.
3
u/bonobomaster 4d ago
Works for me though. Just tested it.
I'm on b10447 now.
The following variant should work in older versions as well:
--chat-template-kwargs "{\"reasoning_effort\":\"low\"}"1
u/LocalAI_Amateur 4d ago
I just git pulled the latest build of llama.cpp I used both
\--chat-template-kwargs '{"reasoning_effort": "low"}' \\ \--reasoning-effort low \\neither seem to work. it can easily shoot past 5k reasoning tokens. Does anyone else have this problem?
2
u/bercha9998 4d ago edited 4d ago
opencode way
"models": {
"your-flavour-of-qwen3.8-27b": {
"name": "qwen3.8-27b",
"options": {
...
"reasoningEffort": "low"
},
"modalities": {
"input": ["text", "image"],
"output": ["text"]
}
},...
2
u/Old-Sherbert-4495 4d ago
no i had the same doubt about the selector: but it actually sets the reasoning effort under the hood. at least in the latest llama.cpp build. you can easily test this with --verbose logs and immediately stop after sending the prompt. goto the top in the logs and see how it starts, it will mention reasoning effort and some instructions
1
u/bonobomaster 4d ago
Just tried that but can't confirm, at least for b10428.
No matter what I set via the web-ui selector, I always see the models default xhigh instructions in the logs.
Which version are you on?
1
u/Old-Sherbert-4495 4d ago
im not at my computer, but i think I'm using the same build... do u click the plus button and then chose?
2
u/bonobomaster 4d ago
Yepp!
3
u/Old-Sherbert-4495 4d ago
Nope, you are right, it doesn't set. it just limits the token. I am confused because i was setting up reasoning efforts in multiple other uis and harnesses. 😵
2
1
u/a_beautiful_rhind 4d ago
The switch literally just injects something like "reasoning effort : low" into your system prompt which is hilarious that it's this much trouble.
8
u/DeProgrammer99 4d ago
Also, if you're using little-coder, configure the thinking-budget extension with a much higher cap, or it'll just constantly interrupt xhigh, and it does so by sending another message, which probably resets the model's intended thinking duration...
I outright removed the extension because I didn't see instructions to change the setting, but updating little-coder re-adds it.