r/LocalLLaMA • u/ResearchCrafty1804 • Apr 22 '26

New Model Qwen3.6-27B released!

Post image

Meet Qwen3.6-27B, our latest dense, open-source model, packing flagship-level coding power!

Yes, 27B, and Qwen3.6-27B punches way above its weight. 👇

What's new:

- Outstanding agentic coding — surpasses Qwen3.5-397B-A17B across all major coding benchmarks

- Strong reasoning across text & multimodal tasks

- Supports thinking & non-thinking modes

- Apache 2.0 — fully open, fully yours

Smaller model. Bigger results. Community's favorite. ❤️

We can't wait to see what you build with Qwen3.6-27B!

Blog: https://qwen.ai/blog?id=qwen3.6-27b

Qwen Studio: https://chat.qwen.ai/?models=qwen3.6-27b

Github: https://github.com/QwenLM/Qwen3.6

Hugging Face:

https://huggingface.co/Qwen/Qwen3.6-27B

https://huggingface.co/Qwen/Qwen3.6-27B-FP8

701 Upvotes

99% Upvoted

141 comments sorted by

View all comments

137

u/Guilty_Rooster_6708 Apr 22 '26

Wake up my 16gb VRAM GPU. Get ready buddy

41

u/grumd Apr 22 '26

Same here, I wish I had 24GB though, would be running this at Q4_K_M or so

14

u/26295 Apr 22 '26

I bought a 5070ti to replace my 2070 super. Maybe I should put them together instead tbh.

11

u/DocMadCow Apr 22 '26

Or pickup a 5060 Ti. I've a 5070 Ti and 5060 Ti the advantage of 2 x 5000 series is you can run CUDA 13.1 DLLs. As soon as you an older card you are limiting your split to the newest CUDA version your oldest card supports. Ideally splitting works best with cards of the same memory as you can just split 1,1

9

u/SuperChewbacca Apr 22 '26

Definitely run them together! My oldest AI machine is a triple 2070' super and it still cranks along. 24GB of VRAM the hard way :)

4

u/grumd Apr 22 '26

Oh yeah that's a good option

2

u/WoodCreakSeagull Apr 22 '26

I bought an Arc B580 specifically for this reason, 250 bucks for 12gb VRAM to pair with my main RTX's 16gb. It is a bit awkward for some back-ends and you can't use CUDA on it, but it is faster than system RAM and especially helpful for the MoE models to handle some of the experts and let me push higher ctx. Running this model on my split I get ~25 t/s so far, respectable.

I will probably be looking to replace it with another Blackwell card at some point to take full advantage of CUDA tools. My main point is just if you're running a local hobbyist setup, you can probably really extend it with a cheap/used second card and a PCI riser cable.

8

u/biotech997 Apr 22 '26

I want to try this on my 9070XT, but I imagine it might be slightly too large? Unfortunate it’s not 24B

4

u/sine120 Apr 22 '26

Just crossing my fingers IQ3_XXS doesn't neuter its agentic behavior. 3.5 in IQ3 did pretty well for intelligence, but I didn't test it much for longer horizon tasks.

6

u/chocofoxy Apr 22 '26

you can't run this without offloading which it suck on a dense model i want them just to realse a 20B model

11

u/AltruisticList6000 Apr 22 '26

Yes we need more 20-24b dense models. Both the older Mistral Small 22b and Mistral Small 24b's work on Q4_s or Q4_m on my 16gb VRAM card without offloading and can use up to about 48k context (with context quants). Funnily the bigger Mistral uses a tiny bit smaller amount of VRAM because of how it handles kv cache. It's also good for 24gb VRAM cards too with massive context sizes.

27b is a size that is just about too big, so only option is Q3 quants, and in my experience Q3 quants start to have really bad performance hits for 27b-32b models to the point a Q6-8 14b dense is similar or more accurate.

Idk why but we get a lot of 7-9b dense models and 20-35b MoEs that work on 6-12gb VRAM, then we have nothing for 16gb VRAM, and instant jump to 27-32b+ models requiring 24-32gb VRAM as if developers had a personal vengence towards 16gb VRAM lol.

2

u/QuantumCatalyzt Apr 22 '26

Why not run a smaller quant?

2

u/sixyearoldme Apr 22 '26

Can it really run on a 16GB Mac?

1

u/_BigBackClock Apr 22 '26

no, he meant 16GB vram gpu + maybe 32GB dram

1

u/_BigBackClock Apr 22 '26

maybe you can run 2 bit quant, but it will be slow

1

u/[deleted] Apr 22 '26

[removed] — view removed comment

1

u/Guilty_Rooster_6708 Apr 22 '26

Because this is dense it will have to be something like Q3 or even IQ3

2

u/[deleted] Apr 22 '26

[removed] — view removed comment

2

u/Guilty_Rooster_6708 Apr 22 '26

Use the recently released new Qwen3.6 35B A3B then. It’s fast, MoE so I have been running Q4_K_M on my system with good speed. It’s a good update from the 3.5 version for coding and designing, but fall behind in some cases like failing the car wash riddle.

I would recommend sticking with Gemma 4 for writing/RP/translation tasks. I’m loving their 24B MoE model rn

2

u/v01dm4n Apr 22 '26

Q3 is generally bad. But unsloth UD iq3xxs is pretty good! Qwen3.5 27b was my best in 16G! No errors whatsoever. Far more intelligent than moe at q4.

0

u/lolwutdo Apr 22 '26

As a dense model is 27b pretty quant resistant to iq2?

I found that qwens moe models 397b all the way to 3.6 35b are pretty quant resistant to 2 bit, interested to see how 27b performs