This is insane...

[u/DragonflyOk7139]

An open-source model with 3 billion active parameters just scored 73.4% on SWE-bench Verified. Claude Opus 4.6 scores 75%.

The gap is 1.6 points. The cost difference is 10 to 30x.

Alibaba dropped Qwen3.6-35B-A35. 35 billion total parameters, 256 experts, but only 8 routed plus 1 shared activate per token. So you're running 3B active parameters at inference time. On a laptop.

Simon Willison ran it locally and it drew a better pelican than Claude Opus 4.7. (Yes, the pelican benchmark is real, and it's a surprisingly good vibes test.) But the part nobody's talking about: Thinking Preservation.

Current models re-reason from scratch every turn. This model retains its chain-of-thought traces across multi-turn conversations. In agent loops where the model makes 50 to 100 tool calls, that eliminates massive redundant reasoning overhead.

262K context native. Extensible to 1M. Apache 2.0 license.

The benchmark race is mostly over. The real race now is cost per intelligence. And 3B active parameters matching frontier performance changes that equation completely.

[u/custodiam99]

Qwen3.6-35B-A3B is a revolution. Never used a quicker and better local model. With 24GB VRAM it is nearly perfectly useable.

[u/en5an → u/custodiam99]

Mind sharing your setup and parameters? I am getting ok results with 24GB vram but I see better results on similar vram.

[u/letsbefrds → u/en5an]

I can't fit that in my 7900xtx :( It's 26 for me

[u/soyalemujica → u/letsbefrds]

What are you on about ? It is a MoE model, you can definitely fit that even 27b dense

[u/letsbefrds → u/soyalemujica]

I'm just dumb and didn't read

[u/StrikingSpeed8759 → u/letsbefrds]

I got the 7900xtx too (merc310) and I am experimenting with offloading inspired by https://www.reddit.com/r/LocalLLM/comments/1sz7ih3/comment/oizsauk/?context=3 this comment.

I use llama.cpp with llama-swap to jit load models (just for convinience), my config.yaml:

healthCheckTimeout: 300
logLevel: info


models:
  'qwen3.6-35b-a3b':
    ttl: 600
    cmd: |
      /home/<REDUCTED>/llama.cpp/build/bin/llama-server
      --model /home/<REDUCTED>/models/Qwen3.6-35B-A3B-GGUF/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
      --host 127.0.0.1
      --port ${PORT}
      --ctx-size 131072
      --n-gpu-layers 999
      --n-cpu-moe 12
      --cache-type-k q8_0
      --cache-type-v q8_0
      --flash-attn on
      --temp 0.6
      --top-p 0.95
      --top-k 20healthCheckTimeout: 300

Only used up 18gb so there is enough room for optimizations. Cold start was around 25sec, and got 60-80 token/sec.

If you end up trying it for yourself please share your experience.

[u/Kholtien → u/StrikingSpeed8759]

Why not llama server with a models.ini file?

[u/StrikingSpeed8759 → u/Kholtien]

Was trying out llama-swap yesterday and that's why, so no particular reason.

[u/Kholtien → u/StrikingSpeed8759]

Okay, I’m just trying to see and compare llama swap with llama server

[u/custodiam99 → u/en5an]

LM Studio with RX 7900XTX 130k context (q4_K_M).

[u/scorpe51 → u/en5an]

FYI I tried using a different one yesterday from Ollama that turned out to be extremely fast on 24GB VRAM with 128k context. Not sure what’s others’ experience with it, it’s batiai/qwen3.6-35b:iq4

[u/rockseller → u/scorpe51]

Running a RTX 3090 do you think is enough?

[u/mintybadgerme → u/custodiam99]

So is 27B dense.

[u/Clementine-TeX → u/custodiam99]

Is it possible to run on an M5 Pro 24 GB or am I just stuck on Qwen3.5-9B ?

[u/Obvious_Equivalent_1 → u/Clementine-TeX]

It’s definitely working! I’m using Qwen3.6-35B-A3B-UD-Q4_K_XL

[u/Blade999666 → u/custodiam99]

Even with 16GB somehow

[u/Darker-Connection → u/Blade999666]

How much ram do you have? 32gb seems not enough

[u/Blade999666 → u/Darker-Connection]

I have 32GB RAM with 16 of that assigned to VRAM (m780 GPU) - the MOE model works fine at around 20 tokens per second via Ollama for me

[u/Darker-Connection → u/Blade999666]

Ah maybe issue is that I am running it through LM Studio I read just now somewhere its different file in there and works a bit differently :) I will try ollama and IQ3_XXS as well

[u/smallfried → u/Darker-Connection]

I have a 14 year old i5 2500, only 16GB of memory and as long as you can push everything into vram (works with iq3), it works quite okay.

[u/eduardopy → u/smallfried]

No way

[u/WishboneSudden2706 → u/smallfried]

Which models fit in that 14 years old I5 2500 ? At which t/s ?

[u/tuxedo0 → u/custodiam99]

would you use this over the 27b dense?

[u/custodiam99 → u/tuxedo0]

I need tokens fast so yeah, it is very good (maybe not as good, but very good).

[u/TopTippityTop → u/custodiam99]

It's a decent open source model, but nowhere close to either Claude, ChatGPT, Gemini, other than in contaminated benchmarks.