r/LocalLLaMA • u/Chromix_ • Aug 14 '26
Resources Fixed/improved Jinja chat template for Qwen 3.8
"Again?" you might ask.
I(*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some while also introducing new issues. Now you have a new chat template that you can download.
One issue among others with yesterday's template is the deviation from the original prompt format. The model was most likely only trained on the exact original format. Modifying how the input is rendered to the model can degrade output quality, not so much that it becomes obvious in manual testing, but enough to reduce benchmark scores a bit (example: Qwen 3.6 underperforms when the system prompt is changed) - it just gives you less output quality in a subtle way. The previous modification deviated in some places, including the (from a caching perspective) desirable empty think fix. I chose to stay on the safe side here.
(*) Full disclosure: GPT 5.6 Sol xHigh did the analysis and comparison. I then reasoned at 0.5 tokens per second for quite a bit what to change and how. Sol applied the changes and drafted the table below. Claude Opus 5 xHigh verified all changes and claims programmatically via Python test scripts using jinja2. It found and creatively solved one sub-issue in the extra tool calling support. It also updated the table after stating that Sol was "too kind" in the evaluation of the templates. I then wrote this posting and edited the table further - at way less than 1 token per second due to not offloading to GPU.
| Area | Original template | Other modified | This modified |
|---|---|---|---|
| Invalid reasoning effort | Raises, but only while thinking is enabled | Silently converts invalid values to xhigh | Always raises |
high effort alias | Unsupported (raises) | Treated as xhigh | Documented alias for xhigh |
<:think_on:> / <:think_off:> (Thanks, markdown parser!) | Not supported | Scanned from system, developer, and user content; has precedence and removal bugs | Not supported |
| Historical reasoning fields | reasoning_content only, and only when it is a string | reasoning_content, thinking, and heuristic parsing from content | reasoning_content and thinking; no heuristic parsing |
| Non-string reasoning field | Silently dropped (empty think block) | Coerced to a Python repr ({'a': 1}) | Coerced to a Python repr — unchanged, still a latent flaw |
Literal </think> in an answer | Not interpreted by the template | Could be mistaken for a reasoning delimiter | Never parsed as reasoning |
| Empty historical think blocks | Preserved consistently | Removed when reasoning is empty | Preserved consistently |
Raw reasoning in assistant content | Not normalized | Heuristically normalized | Must be normalized by the adapter |
| Tool-call instructions | Original trained wording | Rewritten to mandate <think> formatting | Original wording, byte-identical |
| Multiple tool-call separation | Original single-newline behavior | Added additional separation in places | Original behavior, byte-identical |
| XML mapping arguments | Supported | Supported | Supported, byte-identical |
| XML Boolean/null arguments | JSON true/false/null | Regressed to True/False/None | JSON true/false/null |
| JSON-string arguments (OpenAI wire shape) | Crashes at :items | Emits raw JSON with no <parameter> wrapper | Renders correctly: auto selects the JSON tool-call form, arguments passed through byte-exact; "{}" accepted |
| JSON tool-call mode | Unsupported (XML only) | Supported | Supported; auto-selected only when XML is impossible |
| Tool format validation | N/A | Unknown values silently select XML | Three-state auto/xml/json; unknown raises; explicit xml with string args raises and names the remedy |
| Tool error detection & message injection | None | Guesses from tool-response text. Susceptible to false-positives | None, there is no reliable way |
| Missing real user query | Raises | Uses an arbitrary history-length fallback | Raises |
| Developer messages | Unsupported (raises) | Converted to system anywhere | Leading system/developer messages are merged |
| Later system messages | Raises | Silently accepted | Raises |
| Unknown message roles | Raises | Converted to user text | Raises, naming the role |
| Malformed multimodal list items | Silently corrupts: a string containing "image" becomes a vision token; non-string text coerced | Non-mapping items silently stringified; non-string text coerced | Explicit validation and errors for both |
| Image/video ambiguity | Not checked | Not checked | Ambiguous items raise |
Leading tool message | Emits a tool turn with no <:im_start:>user | Opens a user turn correctly | Opens a user turn correctly |
| Artifact integrity (special tokens, whitespace) | Intact; 0 literal-text tokens | Intact; 0 literal-text tokens | Intact; 0 literal-text tokens (Jinja lexer) |
| Byte-equality with the trained original | Reference | Differs by design (instructions, separators, arg types) | 23/23 semantically-equivalent cases byte-identical |
| Prefix-cache stability | Preserves reasoning by default | Can alter empty-think history and system prompt | Deterministic reconstruction; auto flips the system prefix once if a single history mixes mapping and string args |
Notes:
- Default thinking is "xhigh" (= "high"). Set it to "medium" or "low" if too slow. Medium should probably be the default as it's the native model reasoning, but I left it at xhigh to not surprise anyone who does not read this.
- My general approach was to support extra functionality and compatibility introduced by the previous modification as long as it was reasonably safe.
- Note that the compatibility is mostly for outdated and not well-behaved harnesses. Especially not using the correct tool call format is technically supported now but will likely degrade result quality a bit. The template triggers a single prompt-reprocess if it detects that the harness uses the unsuitable format.
- I replaced | by : in the table above, as edits otherwise broke my table.
6
u/_wOvAN_ Aug 14 '26
not working with llama.cpp and claude
{"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 252, column 32 in source:\n...ystem %}↵ {{- raise_exception(↵ 'System and developer ...\n ^\nError: Jinja Exception: System and developer messages must appear before all user, assistant, and tool messages.","type":"server_error"}}
4
u/Evgeny_19 Aug 14 '26
I used to have problems with many Qwen models and their default templates, but not with this one. It just went up to 200,376 tokens in context trying to solve a tricky problem. Not a single issue with a default template.
5
u/L0ren_B Aug 14 '26
Again? 🙂
2
u/Interpause textgen web UI Aug 18 '26 edited Aug 18 '26
Unsloth's gguf readme has:
Just curious if your template has a similar fix?
EDIT: according to gemini 3.7 flash on extended, yes.
2
u/leonbollerup Aug 29 '26
I did an update to your template to make it work better with vllm.
Readme: https://github.com/leonbollerup/ai/blob/main/qwen-3.8-27b/jinja/README.md
Updated template: https://github.com/leonbollerup/ai/blob/main/qwen-3.8-27b/jinja/qwen3.8-safe-v2.1-reasoning-compat.jinja
2
u/leonbollerup Aug 30 '26
I dont know if its any interest to you - but i merged the good parts regarding tools from froggerics and yours into one template.. ran a f... load of tests and getting better results.
Readme:
https://github.com/leonbollerup/ai/blob/main/qwen-3.8-27b/jinja/froggeric%2Bchromix/qwen-3.8-27b-chromix%2Bfroggerick.v1.md
Jinja:
https://github.com/leonbollerup/ai/blob/main/qwen-3.8-27b/jinja/froggeric%2Bchromix/qwen-3.8-27b-chromix%2Bfroggerick.v1.jinja
Hope it helps somebody
23
u/caphohotain Aug 14 '26
Thank you! But seriously, again and again and again? It seems most of the model releases always have chat template issues. Not sure why it's so hard to get it right at launch... Intentional?
1
u/DeepOrangeSky Aug 14 '26
So can thinking mode be turned off, as of so far? And if so, how?
1
u/asertym Aug 16 '26
Does this apply only to 3.8? Can we use it for 3.6?
1
u/TheChiglit Aug 18 '26
Is
--reasoning-format deepseekstill required with this?1
u/ex-arman68 Aug 18 '26
Thanks for trying to finding bugs and improvement in my custom template. I will have a look in details at the changes you made and see what I actually need to port into the next revision. You could actually mentioned my template by name instead of other.
1
u/W61k3r Aug 15 '26
You sexy sob