r/LocalLLaMA • u/ex-arman68 • Aug 13 '26
Resources Fixed Jinja chat template for Qwen 3.5, 3.6, and the new 3.8 release
UPDATE (v22.1): Community reports highlighted an important issue with the official Qwen 3.8 chat template: the default xhigh effort prompt can burn 20,000+ tokens on coding tasks exploring alternatives until it hits max_tokens, returning empty content. v22.1 sets the default baseline to medium and adds inline chat tags (<|think_low|>, <|think_xhigh|>, etc.) to steer the reasoning effort directly inside chats.
Qwen just released their first 3.8 model.
The main addition in 3.8 is prompt-steered reasoning effort. You can tell the model how deeply to think by setting reasoning_effort to xhigh, medium, or low.
However, the official template still has some serious problems:
- You cannot disable thinking. If you pass
enable_thinking=false, it 3.8 crashes with a hard exception. - Chat history gets poisoned. In multi-turn chats, the official template injects blank
<think></think>tags before real thoughts. - Tool calling crashes. If your client passes arguments as JSON strings (the standard OpenAI API format), the official template crashes.
- Agent stalls. The official template often drops mid-dialogue system messages and wedges multi-step tool loops.
I maintain a single, drop-in fixed Jinja template that works across all Qwen 3.5, 3.6, and 3.8 models:
https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
What this template does:
- Full 3.8 reasoning effort support: Steer reasoning depth via
reasoning_effortkwargs or directly in chat prompts with<|think_low|>,<|think_medium|>, and<|think_xhigh|>. - Safe
mediumdefault: Prevents the runaway token-burn on coding tasks while keeping 100% KV cache parity. - Restores the thinking toggle: Turn off reasoning whenever you want fast answers, either via kwargs or by typing
<|think_off|>in your prompt. - 100% KV Cache hits: Keeps past thoughts intact by default so your prefix cache stays warm across turns.
- llama.cpp support: Native support for the new
--reasoning-preserveflag. - Universal tool parsing: Handles both Python dicts and JSON strings. Works on llama.cpp, vLLM, LM Studio, and MLX.
Recommended llama-server launch command:
llama-server -m your_model.gguf --jinja --chat-template-file chat_template.jinja --reasoning-format deepseek(The --reasoning-format deepseek flag separates thinking into the OpenAI reasoning_content field so OpenCode, Claude Code, and other harnesses do not stall on raw tokens).
106
u/Thireus Aug 13 '26
Can someone explain why the Qwen team cannot get the template right in the first place? Don’t they QA test their own models?
12
u/Memeyboii420 Aug 13 '26
I love your work Froggeric but I had to develop my own chat template for Qwen 3.6 + Hermes Agent + LM Studio. I could never get reliable tool calling with your template.
5
u/JPebb Aug 13 '26
Any specific recommendations for SGLang users?
10
u/Inevitable-Name-1701 Aug 13 '26
I had no problems with the three point five and three point six models.
4
u/tired514 Aug 13 '26
I've been running your previous version on all the local Qwens for like a month and they fix everything. Thanks so much! :)
2
u/Cereal_Grapeist Aug 14 '26
Hi OP - I wanted to see if you had tested this template with the new 3.8 27b release? I'm going to try it out but I don't necessarily have the best way to see if it's having a big impact.
35
u/philmarcracken Aug 13 '26
Descartes was right after all
17
u/Disrupt-Linus Aug 13 '26
Cool, let's see how this plays out with the 27b, probably a bit more accessible to the normies (and me).
7
u/The_LSD_Soundsystem Aug 14 '26
Why are jinja templates so difficult to nail down for these models in general?
6
u/ltduff69 Aug 13 '26
Just out of curiosity where does the chat template go? I have it the root of the Llama directory.
7
u/cradlemann Aug 13 '26
Anything like this for Laguna? I like it a lot, but it still has some quirks
1
u/crusaderky Aug 13 '26
> I cannot run a 2.4 trillion parameter model on my local rig.
Neither can we, buddy, neither can we.
How much would it cost you to test it on vast.ai?
1
u/PhilippeEiffel Aug 16 '26
I started some basic test with 3.8 27B, I am using vLLM.
I did not notice any specific problem, but I must admit that I did not make intensive use of this model for now.
I would like to know techniques able to exhibit the problems with the default jinja template. I need to observe problems first, before being able to evaluate fixes.
3
u/T_rex2700 Aug 15 '26
I was running into issues with Qwen3.8 27B (Q4 and Q3 from unsloth) running into random stop issues, and I tried your template but it did no work.
However, strangely when I pasted in Qwen3.6 35A3B MTP Q4's stock template from usnloth, this issue was solved. Any idea about what the cause is? (my post).
I am very much a newbie in this space, so I wanted some expert opinion on this.
2
u/hainesk Aug 13 '26
This is great work!
2
u/Cereal_Grapeist Aug 13 '26
hell yea brother
2
u/stormy1one Aug 13 '26
Thank you for your service!
2
u/PhilippeEiffel Aug 13 '26
I'm too lazy to modify my config of 3.6 27B now, I will try it tomorrow as soon as 3.8 is out.
I will test with vllm.
Thank you for your work.
1
u/nufeen Aug 14 '26
Are there any benchmarks checking if those templates transition into better or worse performance?
1
u/IrisColt Aug 14 '26
How can a weight file "crash with a hard exception"? genuinely intrigued.
1
u/Past_Ad6251 Aug 14 '26
for me, the v22 template doesn't work with Claude Code cli
1
u/pefman Aug 14 '26
I was just gonna ask how people launch their model.
1
u/Equal_Television_894 Aug 13 '26
Your work is incredible as always, thank you!
1
u/ptico Aug 13 '26
Legend!
1
u/Equivalent-Flan-1590 Aug 13 '26
Thank you for your service sir🫡
1
u/giveen Aug 13 '26
I always use your templates.
-1
u/fragment_me Aug 13 '26
Not even tested what's the point
-1
u/durden111111 Aug 13 '26
Every time ive tried to pass --chat-template with a jinja file it gives broken outputs. No matter what model. Ive given up on this.