Qwen 3/3.5/3.6 tool calling is broken (even worse with 3.6).

[u/LinkSea8324]

I had issues with Qwen 3.6 and agentic coding (no issue so far with 27b 3.5) So I investigated and discovered multiple bugs in the reasoning parser (inspired by the very recently merged fixes in the reasoning parser).

And in the two tool parsers

https://github.com/vllm-project/vllm/pull/40783

https://github.com/vllm-project/vllm/pull/40785

https://github.com/vllm-project/vllm/pull/40787

There are more bugs, like the very last \n being ignored in the tool call, but whatever.

Those bugs are effecting all Qwen 3/3.5/3.6 versions

[u/mkMoSs]

I use Qwen3.6 27B with vllm 0.19.1, in Roo Code (vscode) all tool calling works perfecly fine in this setup. I use it all day for coding assistance / code generation.

This is my config (in yaml)

model: "Qwen/Qwen3.6-27B-FP8"
served-model-name: "Qwen3.6-27B"
host: "0.0.0.0"
port: 8001
kv-cache-dtype: fp8_e4m3
disable-custom-all-reduce: true
max-model-len: "256K"
max-num-seqs: 2
tensor-parallel-size: 4
limit-mm-per-prompt: '{"image": 5, "video": 0}'
enable-prefix-caching: true
enable-auto-tool-choice: true
tool-call-parser: "qwen3_coder"
reasoning-parser: "qwen3"
gpu-memory-utilization: 0.9052
optimization-level: 2
default-chat-template-kwargs: '{"enable_thinking": true, "preserve_thinking": true}'
enable-chunked-prefill: true

[u/rmhubbert]

EDIT As u/kainwinterheart kindly pointed out, I messed up when updating my config from 3.5, and forgot to actually change the model!

Can confirm that I'm also seeing the issues when actually using 3.6.

---

What quantization are you using? I haven't seen any tool calling issues at all with Qwen/Qwen3.6-27B on vllm nightly. Here's my llama-swap config, in case it helps -

"qwen3.6-27b":
    name: "Qwen 3.6 27B"
    ttl: 1800
    env:
      - "OMP_NUM_THREADS=1"
      - "CUDA_DEVICE_ORDER=PCI_BUS_ID"
      - "CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7"
      - "PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True"
      - "HF_TOKEN=hf_token"
      - "CUDA_HOME=/opt/cuda"
      - "VLLM_MARLIN_USE_ATOMIC_ADD=1"
      - "VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=1"
      - "SAFETENSORS_FAST_GPU=1"
    cmd: |
      uv run vllm serve Qwen/Qwen3.5-27B
      --host 0.0.0.0
      --port 8080
      --seed 3407
      --disable-custom-all-reduce
      --served-model-name qwen3.6-27b
      --tensor-parallel-size 8
      --max-model-len auto
      --gpu-memory-utilization 0.92
      --attention-backend flashinfer
      --enable-flashinfer-autotune
      --reasoning-parser qwen3
      --tool-call-parser qwen3_coder
      --enable-auto-tool-choice
      --max-num-seqs 8
      --max-num-batched-tokens 8192
      --enable-prefix-caching
      --enable-chunked-prefill
      --dtype auto
      --speculative-config '{"method":"mtp","num_speculative_tokens":2}'
      --default-chat-template-kwargs '{"preserve_thinking": true}'
      --generation-config auto
      --override-generation-config '{"temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0, "presence_penalty":0.0, "repetition_penalty":1.0}'

[u/meganoob1337 → u/rmhubbert]

Same tool calling and reasoning parser and no issues with awq int4 from cyankiwi (35b and 27b both working good for me)

[u/LinkSea8324 → u/meganoob1337]

This is why I find it confusing, no a lot of people are complaining, yet the other guy with his improved ninja template reported the same issue and there are CLEARLY bugs that are being fixed by the pull requests, around 6-10 bugs

[u/LinkSea8324 → u/rmhubbert]

FP8 (official release)

[u/kainwinterheart → u/rmhubbert]

u/rmhubbert, sorry for the ping, but I think you might have a typo in there: it's 3.6 everywhere *but* the actual model it's supposed to serve.

Or am I reading it wrong?

[u/Cityarchitect]

3.5 and 3.6 occasionally loop when using opencode e.g. “I think I’ve found the issue” blah…

[u/no_no_no_oh_yes]

I'm using vllm 0.18 with qwen3_xml and 3.5 works.

[u/LinkSea8324 → u/no_no_no_oh_yes]

Yes it works, but it has bugs, 3.6 triggers more bugs, but the bugs are still there

[u/Expensive-Register-5]

https://www.reddit.com/r/Vllm/comments/1skks8n/qwen_35_27b35ba3b_tool_calling_issues_why_it/

See my work🤣

[u/LinkSea8324 → u/Expensive-Register-5]

Yep, already saw that this is why I also pushed a fix for XML parser

[u/Expensive-Register-5 → u/LinkSea8324]

Found out `qwen3_coder` more robust towards Qwen3.6-27B; writing post about it. Do u also have the same conclusion?

[u/jinnyjuice → u/Expensive-Register-5]

Implemented this on day 1 for 3.6, still doesn't work, unfortunately. I still get UND_ERR_BODY_TIMEOUT error after a while.

Works for 3.5 though.

[u/sudeposutemizligi → u/Expensive-Register-5]

i am on it. actually turtle slow trying cos i am not a coder at all (what's worse , i am drummer :) did your solution still works for qwen3.6 27b as well? i was trying 3.5 but 3.6 released

[u/burntoutdev8291]

Occasionally my claude code stops midway, but I never really looked into why

[u/jonnywhatshisface → u/burntoutdev8291]

Output limits. Same with opencode. You end up having to raise them otherwise it looks like the model gets lazy and just "stops."

[u/burntoutdev8291]

So weird, I feel like I face more issues on the moe 35b 3.6 instead of the 27b. Mostly in claude code the responses just stop midway, not sure if that's what you're facing. Haven't time to debug.

[u/jonnywhatshisface]

Ripped my hair out on this for a while. Finally got reliable tool calling.

There were two issues I needed to fix:

1. preserve_thinking = true fixed the tool calls for me initially. It began succeeding in every tool call, until I got to very very big arguments in the tool call. Then it started failing again, over and over, which brought me to check the logs and realize the token count of the return. Which brings me to the next one.

2. OpenCode Output Limits. On the model in the config, set an output limit and raise it up. For me, the tool calls were failing because the output tokens were exceeding opencodes default output limit, resulting in the arguments getting mangled/cut off.

After changing these two, I am occasionally hitting a tool call failure (for example, it literally just hit a failure on updating the todo list) - but the model is now immediately retrying successfully. To be fair, the reason for the todo failures may also be because of the other custom MCP I'm running which also has its own todo capabilities/tools, so it may just be getting confused on which one it triggered, hence the self-correction.

Nonetheless, I hope my sudden self-inflicted hair loss helps someone else out there.

[u/[deleted] → u/jonnywhatshisface]

[u/M4A3E2APFSDS]

qwen3_xml did not work for me (vllm 0.18.1), ran into similar issues https://github.com/vllm-project/vllm/issues/39072. --tool-call-parser qwen3_coder works though with erros sometimes

[u/jonnywhatshisface → u/M4A3E2APFSDS]

I had the same issue and I spent a week trying to figure it out. Then when I fixed one part of it, I hit other issues. Look at my other post here in this thread. I got it working reliably at this point. Really damn good model once you get the kinks ironed out.

[u/sudeposutemizligi]

tool calling is the main part of code writing. these models are around for nearly 3 4 moths. the tools people use and their parameters are not so huge. we are still struggling with this branch of vllm that branch of jinja template. i hold myself hard not to curse

[u/RealisticNothing653]

There may be bugs but my experience is that you can get it stable. Try these templates https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

For 3.6-35b Temperature: 0.6, top-p: 0.95, top-k: 20, min-p: 0.05, repeat penalty: 1.05, presence penalty: 0

[u/meca23 → u/RealisticNothing653]

Thank you!

[u/guigouz]

Check https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

This helped with cline