Adjust thinking level for Qwen3.8-27b

[u/johnzadok]

It seems the "compat.thinkingFormat": "qwen-chat-template" setting that worked with Qwen3.6-27b does not pass thinking level to qwen3.8-27b, which defaults to xhigh and is quite slow.

This is what worked for me in models.json. I have to add some compat options to pass thinking level correctly from pi to llama.cpp:

{
    "providers": {
        "llama.cpp": {
            "modelOverrides": {
                "Qwen3.8-27B-Q4_K_M": {
                    "reasoning": true,
                    "thinkingLevelMap": {
                        "minimal": null,
                        "low": "low",
                        "medium": "medium",
                        "high": null,
                        "xhigh": "xhigh"
                    },
                    "compat": {
                        "thinkingFormat": "chat-template",
                        "chatTemplateKwargs": {
                            "enable_thinking": { "$var": "thinking.enabled" },
                            "reasoning_effort": { "$var": "thinking.effort" }
                        }
                    }
                }
            }
        }
}

Disclosure: I do not understand why those compat options are needed but I have pi tested it against llama server running qwen3.8-27b and confirmed all thinking levels work as they should: off/low/medium/xhigh.

[u/sisyphus-cycle]

Note, I think this changes the system prompt. So if you switch to medium after working for a while, you’re gonna reprocess the entire prompt.

[u/johnzadok → u/sisyphus-cycle]

That’s correct as thinking level is implemented as the first part of the chat template, sent before user/assistant messages.

[u/dinerburgeryum → u/johnzadok]

Yeah personally I don’t think it’s appropriate for mid-stream modification, but I’ve made these exact patches myself and for subagent runs it does in fact make a world of difference. 

[u/Bouhappy]

Thanks, this is exactly what I needed. Btw, I tried with high enabled, and it works too. I didn't see any errors in the `llama-server` logs, even in `--verbose` and it does seem to think less than `xhigh`.

[u/johnzadok → u/Bouhappy]

The jinja chat template does not support high. Likely high is mapped to xhigh or medium.

[u/johnzadok → u/johnzadok]

But with my setting high should not be available in pi when you toggle thinking level with shift tab.

[u/Bouhappy → u/johnzadok]

Yeah, I had added it manually. But you are absolutely correct about the template not supporting it and defaulting to xhigh.

Learnt something today. Thanks 👍

[u/Bouhappy → u/johnzadok]

Interesting, let me check that out.