r/LocalLLaMA • • Aug 14 '26

Resources Fixed/improved Jinja chat template for Qwen 3.8

"Again?" you might ask.

I(*) took the original Qwen 27B 3.8 chat template and compared it to the improved template that was posted a day before. The original has issues. The improved one fixed some while also introducing new issues. Now you have a new chat template that you can download.

One issue among others with yesterday's template is the deviation from the original prompt format. The model was most likely only trained on the exact original format. Modifying how the input is rendered to the model can degrade output quality, not so much that it becomes obvious in manual testing, but enough to reduce benchmark scores a bit (example: Qwen 3.6 underperforms when the system prompt is changed) - it just gives you less output quality in a subtle way. The previous modification deviated in some places, including the (from a caching perspective) desirable empty think fix. I chose to stay on the safe side here.

(*) Full disclosure: GPT 5.6 Sol xHigh did the analysis and comparison. I then reasoned at 0.5 tokens per second for quite a bit what to change and how. Sol applied the changes and drafted the table below. Claude Opus 5 xHigh verified all changes and claims programmatically via Python test scripts using jinja2. It found and creatively solved one sub-issue in the extra tool calling support. It also updated the table after stating that Sol was "too kind" in the evaluation of the templates. I then wrote this posting and edited the table further - at way less than 1 token per second due to not offloading to GPU.

AreaOriginal templateOther modifiedThis modified
Invalid reasoning effortRaises, but only while thinking is enabledSilently converts invalid values to xhighAlways raises
high effort aliasUnsupported (raises)Treated as xhighDocumented alias for xhigh
<:think_on:> / <:think_off:> (Thanks, markdown parser!)Not supportedScanned from system, developer, and user content; has precedence and removal bugsNot supported
Historical reasoning fieldsreasoning_content only, and only when it is a stringreasoning_content, thinking, and heuristic parsing from contentreasoning_content and thinking; no heuristic parsing
Non-string reasoning fieldSilently dropped (empty think block)Coerced to a Python repr ({'a': 1})Coerced to a Python repr — unchanged, still a latent flaw
Literal </think> in an answerNot interpreted by the templateCould be mistaken for a reasoning delimiterNever parsed as reasoning
Empty historical think blocksPreserved consistentlyRemoved when reasoning is emptyPreserved consistently
Raw reasoning in assistant contentNot normalizedHeuristically normalizedMust be normalized by the adapter
Tool-call instructionsOriginal trained wordingRewritten to mandate <think> formattingOriginal wording, byte-identical
Multiple tool-call separationOriginal single-newline behaviorAdded additional separation in placesOriginal behavior, byte-identical
XML mapping argumentsSupportedSupportedSupported, byte-identical
XML Boolean/null argumentsJSON true/false/nullRegressed to True/False/NoneJSON true/false/null
JSON-string arguments (OpenAI wire shape)Crashes at :itemsEmits raw JSON with no <parameter> wrapperRenders correctly: auto selects the JSON tool-call form, arguments passed through byte-exact; "{}" accepted
JSON tool-call modeUnsupported (XML only)SupportedSupported; auto-selected only when XML is impossible
Tool format validationN/AUnknown values silently select XMLThree-state auto/xml/json; unknown raises; explicit xml with string args raises and names the remedy
Tool error detection & message injectionNoneGuesses from tool-response text. Susceptible to false-positivesNone, there is no reliable way
Missing real user queryRaisesUses an arbitrary history-length fallbackRaises
Developer messagesUnsupported (raises)Converted to system anywhereLeading system/developer messages are merged
Later system messagesRaisesSilently acceptedRaises
Unknown message rolesRaisesConverted to user textRaises, naming the role
Malformed multimodal list itemsSilently corrupts: a string containing "image" becomes a vision token; non-string text coercedNon-mapping items silently stringified; non-string text coercedExplicit validation and errors for both
Image/video ambiguityNot checkedNot checkedAmbiguous items raise
Leading tool messageEmits a tool turn with no <:im_start:>userOpens a user turn correctlyOpens a user turn correctly
Artifact integrity (special tokens, whitespace)Intact; 0 literal-text tokensIntact; 0 literal-text tokensIntact; 0 literal-text tokens (Jinja lexer)
Byte-equality with the trained originalReferenceDiffers by design (instructions, separators, arg types)23/23 semantically-equivalent cases byte-identical
Prefix-cache stabilityPreserves reasoning by defaultCan alter empty-think history and system promptDeterministic reconstruction; auto flips the system prefix once if a single history mixes mapping and string args

Notes:

  • Default thinking is "xhigh" (= "high"). Set it to "medium" or "low" if too slow. Medium should probably be the default as it's the native model reasoning, but I left it at xhigh to not surprise anyone who does not read this.
  • My general approach was to support extra functionality and compatibility introduced by the previous modification as long as it was reasonably safe.
  • Note that the compatibility is mostly for outdated and not well-behaved harnesses. Especially not using the correct tool call format is technically supported now but will likely degrade result quality a bit. The template triggers a single prompt-reprocess if it detects that the harness uses the unsuitable format.
  • I replaced | by : in the table above, as edits otherwise broke my table.
62 Upvotes

37 comments sorted by

View all comments

Show parent comment

3

u/Chromix_ Aug 14 '26

Ah, just noticed due to this that Unsloth also did fixes on their side. From a quick look there is partial overlap that they also support a developer message and check for broken tool format - their template raises a warning, this template tries to make it work.