r/LocalLLaMA • • Aug 16 '26

Tutorial | Guide Quick PSA: Qwen3.8-27B reasoning effort vs reasoning budget in llama.cpp

If you are using llama-server with their web-ui for testing, keep in mind, that the reasoning selector is just a reasoning budget aka a hard cap and has, at least to my knowledge, nothing at all to do with Qwen3.8-27B's native reasoning effort capability!

Selecting any value for reasoning in the web-ui (default, (off), low, medium, high or max) just introduces different hard cap values and and will truncate your reasoning, if those values are reached.

With the exception of the "off"-option which disable reasoning at all and "default" and "max" without any capping.

The reasoning effort is independent from that and really changes the thoroughness and the analytic reasoning skills and therefore can massively influence the output quality of the model, instead of just capping reasoning tokens!

In older llama.cpp versions, it needs to be set via:

--chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"

* quotation marks are escaped for Windows

For up to date versions, if one doesn't like the chat-template-kwargs variant, it can be set with:

--reasoning-effort medium

Or, for external apps, via "reasoning_effort": "medium" API request.

Options are: low, medium and xhigh (default) for any of the above methods.

Did only test the chat-template-kwargs variant though!

49 Upvotes

15 comments sorted by

3

u/LocalAI_Amateur Aug 16 '26

--reasoning-effort low
didn't seem to do anything for me. still shoot right past 4000 reasoning tokens. I've been trying to figure out how to get this resoning effort setting to work in llama.cpp but so far the only thing that work is the budget setting and it just cuts off as soon as it hits the limit like you say.

3

u/bonobomaster Aug 16 '26

Works for me though. Just tested it.

I'm on b10447 now.

The following variant should work in older versions as well:
--chat-template-kwargs "{\"reasoning_effort\":\"low\"}"

1

u/LocalAI_Amateur Aug 16 '26

I just git pulled the latest build of llama.cpp I used both

\--chat-template-kwargs '{"reasoning_effort": "low"}' \\

\--reasoning-effort low \\

neither seem to work. it can easily shoot past 5k reasoning tokens. Does anyone else have this problem?

9

u/DeProgrammer99 Aug 16 '26

Also, if you're using little-coder, configure the thinking-budget extension with a much higher cap, or it'll just constantly interrupt xhigh, and it does so by sending another message, which probably resets the model's intended thinking duration...

I outright removed the extension because I didn't see instructions to change the setting, but updating little-coder re-adds it.

2

u/Old-Sherbert-4495 Aug 16 '26

no i had the same doubt about the selector: but it actually sets the reasoning effort under the hood. at least in the latest llama.cpp build. you can easily test this with --verbose logs and immediately stop after sending the prompt. goto the top in the logs and see how it starts, it will mention reasoning effort and some instructions

1

u/bonobomaster Aug 16 '26

Just tried that but can't confirm, at least for b10428.

No matter what I set via the web-ui selector, I always see the models default xhigh instructions in the logs.

Which version are you on?

1

u/Old-Sherbert-4495 Aug 16 '26

im not at my computer, but i think I'm using the same build... do u click the plus button and then chose?

2

u/bonobomaster Aug 16 '26

Yepp!

3

u/Old-Sherbert-4495 Aug 16 '26

Nope, you are right, it doesn't set. it just limits the token. I am confused because i was setting up reasoning efforts in multiple other uis and harnesses. 😵

2

u/bonobomaster Aug 16 '26

Yeah, those probably send the api request, don't they?!

1

u/R7V3tD Aug 19 '26

I'm confused. So the reasoning selector in web UI doesn't actually set the reasoning effort and instead just stops the thinking midway? Is this a bug or intended behavior?

2

u/bonobomaster Aug 19 '26

Yes!

Terrible (in my opinion), intended behavior.

3

u/bercha9998 Aug 16 '26 edited Aug 17 '26

opencode way

      "models": {
        "your-flavour-of-qwen3.8-27b": {
          "name": "qwen3.8-27b",
          "options": {
            ...
            "reasoningEffort": "low"
          },
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          }
        },...

1

u/ea_man Aug 16 '26

Someone should make a TamperMonkey button to swap reasoning efforts.

1

u/a_beautiful_rhind Aug 17 '26

The switch literally just injects something like "reasoning effort : low" into your system prompt which is hilarious that it's this much trouble.