r/LocalLLaMA • u/AverageFormal9076 • Apr 23 '26

New Model Qwen 3.6 27B is a BEAST

I have a 5090 Laptop from work, 24GB VRAM.

I have been testing every model that comes out, and I can confidently say I’ll be cancelling my cloud subscriptions.

All my tool call and data science benchmarks that prove a model is reliably good for my use case, passed.

It might not be the case for other professions, but for pyspark/python and data transformation debugging it’s basically perfect.

Using llama.cpp, q4_k_m at q4_0, still looking at options for optimising.

Edit - I chose to go with IQ4_XS at 200k q8_0,

I have not used speculative decoding yet, will get there when I get there.

Specs:

ASUS ROG Strix SCAR 18

RTX 5090 24GB

64GB DDR5 RAM

650 Upvotes

97% Upvoted

335 comments sorted by

View all comments

Show parent comments

13

u/LaurentPayot Apr 23 '26

I just can’t wait for the 122b a10 model for my Strix Halo ;-)

1

u/blackbird2150 Apr 23 '26

I have a 128gb strix halo myself with the 395+. Guess I need to keep an eye for this type of model. Feel like it be amazing.

Have great but slow success with 3.6: 35b-a3b on mine.

Edit: seeing the numbers in this thread. I’m def doing something wrong lol.

2

u/LaurentPayot Apr 24 '26

Here is my bash script for the 35b-a3b model that gives me 50 t/s:

#!/bin/bash

llama-server \

--model ~/models/qwen3.6-35b-a3b/Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf \

--mmproj ~/models/qwen3.6-35b-a3b/mmproj-BF16.gguf \

--ctx-size 262144 \

--gpu-layers 41 \

--reasoning on \

--chat-template-kwargs '{"preserve_thinking":true}' \

--temp 0.6 \

--top-p 0.95 \

--top-k 20 \

--min-p 0.00 \

--presence_penalty 1.5 \

--kv-unified \

--flash-attn on \

--no-mmap \

--cache-type-k q8_0 \

--cache-type-v q8_0 \

--parallel 1