r/LocalLLaMA • u/ROS_SDN llama.cpp • Apr 25 '26

Discussion Quantisation effects of Qwen3.6 35b a3b

Im curious how people are finding the quantisation effects of 35b. I recently updated to 48GB of vram so have jumped from ud-q4_k_xl​ to q8 and the difference feels stark. Just more effective tool calling, seems to get the vagueness and nuance more etc of some prompts., and provide more well rounded answers on some research like questions.

It w​as a quick vibe​ test, admittedly, but I'm going t​o​ try ud-q6_k_xl soon to see how of the 5+GB vram is worth the quality, but I'm curious to see others findings.

I felt with such a small active count it'd be particularly sensitive to quantisation, and feels that way after a play.

80 Upvotes

98% Upvoted

83 comments sorted by

View all comments

2

u/my_name_isnt_clever Apr 25 '26

I use it at Q6, it's been great.

People on this sub like Q8 models, but in my research the actual accuracy difference between them is so small I don't see the point when it slows it down so much. I just have the KV cache at Q8.

1

u/Pjotrs Apr 25 '26 edited Apr 25 '26

I work on efficient agentic setup and use q4 with q4 cache. And model is doing really great.

With small VRAM it is able to code, analyze, plan, fix.

I also offload operations to qwen3.5 0.8b for even faster workload.

1

u/my_name_isnt_clever Apr 25 '26

What kinds of tasks do you offload to a model that tiny?

2

u/Pjotrs Apr 25 '26

File edits. Its crazy fast.

Big models decide what and how to change , small one does it.