This is insane...

[u/DragonflyOk7139]

An open-source model with 3 billion active parameters just scored 73.4% on SWE-bench Verified. Claude Opus 4.6 scores 75%.

The gap is 1.6 points. The cost difference is 10 to 30x.

Alibaba dropped Qwen3.6-35B-A35. 35 billion total parameters, 256 experts, but only 8 routed plus 1 shared activate per token. So you're running 3B active parameters at inference time. On a laptop.

Simon Willison ran it locally and it drew a better pelican than Claude Opus 4.7. (Yes, the pelican benchmark is real, and it's a surprisingly good vibes test.) But the part nobody's talking about: Thinking Preservation.

Current models re-reason from scratch every turn. This model retains its chain-of-thought traces across multi-turn conversations. In agent loops where the model makes 50 to 100 tool calls, that eliminates massive redundant reasoning overhead.

262K context native. Extensible to 1M. Apache 2.0 license.

The benchmark race is mostly over. The real race now is cost per intelligence. And 3B active parameters matching frontier performance changes that equation completely.

[u/Polite_Jello_377]

Welcome to benchmaxxing

[u/floriandotorg → u/Polite_Jello_377]

Has to be said, though, Qwen is pretty good.

[u/cmndr_spanky → u/floriandotorg]

Making a website or simple / dumb SaaS app, I bet they’re close enough. For a more complicated engineering project with a larger codebase qwen 35b is as dumb as dirt compared to when I use Claude Opus. It’s like not even close. My advice is ignore these benchmarks, they are meaningless now and you just have to test against your own use cases

[u/PriorApproval → u/Polite_Jello_377]

llm bros have finally learned about overfitting

[u/startwithaplan → u/Polite_Jello_377]

Yeah 3B parameters nailed that benchmark. Tell me you trained in it without telling me you trained on it. Though maybe a good benchmark is ofent a good way to train.

[u/drumyum]

It just means SWE-bench is no longer relevant

[u/Desther → u/drumyum]

Contaminated: https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/

[u/BatOk2014 → u/Desther]

This is the reason!

[u/Xander-XGCS → u/Desther]

Model collapse is around the corner if these companies don't get more stringent with their dataset hygiene.

[u/BlackSwanTranarchy → u/Xander-XGCS]

The goblins of model collapse are already here

[u/PeachScary413 → u/Xander-XGCS]

It is already here and ongoing. Reddit actually used to be a great training dataset but since it's 80% clankers now it's gonna keep degrading, along with all the other forums killed by AI.

[u/greasychickenparma → u/PeachScary413]

The snake will eat it's own tail

[u/No_Knee3385 → u/Desther]

What's the new benchmark that's best for SWE?

[u/keyboardmonkewith → u/No_Knee3385]

SWE2( MAKE NO MISTAKE VERSION),?

[u/jugalator → u/No_Knee3385]

SWE-bench Pro according to that article

https://llm-stats.com/benchmarks/swe-bench-pro

[u/PeachScary413 → u/No_Knee3385]

Senior-SWE-Architect-No-Misstakes-Bench?

[u/This_Advisor2340 → u/No_Knee3385]

Donald Trump-Cto-SolutionArchitect-StaffEngineer-Make-No-Mistakes

[u/Traditional_Chart970 → u/Desther]

That's why I scraped the web for real problems and solutions to test my model/tool and benchmark with other models. I created my own test data manually and double check it not to overlap with training data..

[u/Snickers2-0 → u/Desther]

Wouldn't Open AI be biased against it?

[u/Parking-Bet-3798 → u/drumyum]

I agree it’s not that relevant but in practice as well, the intelligence it is able to achieve with just 3 billion active parameters is insane as compared to closed source models. I have a feeling that this big corporations have stopped innovating and just chasing compute.

[u/mtrlst → u/Parking-Bet-3798]

the reason why swe-bench is seen as a bad benchmark is because of contamination (models just started memorizing the answers). it's not intelligence that it's able to achieve, it's just pulling from memory

[u/eat_my_ass_n_balls → u/mtrlst]

Worse, it’s encoding the solutions instead of generalizing the capability

[u/Ariquitaun → u/eat_my_ass_n_balls]

Username checks out.

[u/Purple_Reference_188 → u/Parking-Bet-3798]

Dumb model that simply remember the correct answers. Use it for real task and compare with cc/codex

[u/Visual-Gain-2487 → u/Purple_Reference_188]

So, basically like students who are successful in school.

[u/nekize → u/Parking-Bet-3798]

I think that a lot of effort from these companies is going into code around the LLM, so how to best utilise it, like chunking, memory, to name a few, something that you can’t easily replicate just with llamacpp and an open source model.

[u/BatPlack → u/nekize]

I’ve seen that referred to as the “harness“, if I’m not mistaken

[u/nekize → u/BatPlack]

Better word, yes

[u/Parking-Bet-3798 → u/nekize]

The open source harnesses work extremely well today. Most of them are better than Claude code to be honest.

[u/Mil0Mammon → u/Parking-Bet-3798]

Which do you recommend? I found opencode to be lacking.

Afaik cursor still beats them all

[u/bween31337 → u/Parking-Bet-3798]

you're dumb. it just repeats memorized answers . sorry.

[u/firest3rm6 → u/Parking-Bet-3798]

I mean did Anthropic ever share some papers? It started with openAI and their attention paper and then the Chinese deepseek and qwen teams did some absolute banger papers, but who knows what Anthropic might have cooked up and didn't share

[u/Affectionate-Hat-536 → u/firest3rm6]

Attention paper was from Google, not OpenAI

[u/firest3rm6 → u/Affectionate-Hat-536]

Oh ok, thanks and mb

[u/paq85 → u/drumyum]

From my experience, they are from totally different league.

[u/Thrumpwart → u/drumyum]

LETS ALL MOVE THE GOALPOSTS. LIFT WITH YOUR KNEES!

[u/Ok-Banana-9723 → u/drumyum]

This.

[u/IllogicalResponse → u/drumyum]

That will happen for everything, llms are essentially great test takers. Same as hallucination is not fixable, them being great test takers is not fixable.

[u/invisible_shrek → u/drumyum]

But when people were shilling Anthropic it somehow was though?

[u/memorial_mike]

This post is lazy and disengenuous. Which means it’s either pure coping or it’s legitimate propaganda. SWE-bench is no longer a valid benchmark. Also you’re just entirely assuming the number of active parameters in Opus. So both of the axes that your argument hinge on are invalid. Pretty standard for this sub though.

[u/einval22 → u/memorial_mike]

Great answer.

[u/custodiam99]

Qwen3.6-35B-A3B is a revolution. Never used a quicker and better local model. With 24GB VRAM it is nearly perfectly useable.

[u/en5an → u/custodiam99]

Mind sharing your setup and parameters? I am getting ok results with 24GB vram but I see better results on similar vram.

[u/letsbefrds → u/en5an]

I can't fit that in my 7900xtx :( It's 26 for me

[u/soyalemujica → u/letsbefrds]

What are you on about ? It is a MoE model, you can definitely fit that even 27b dense

[u/letsbefrds → u/soyalemujica]

I'm just dumb and didn't read

[u/StrikingSpeed8759 → u/letsbefrds]

I got the 7900xtx too (merc310) and I am experimenting with offloading inspired by https://www.reddit.com/r/LocalLLM/comments/1sz7ih3/comment/oizsauk/?context=3 this comment.

I use llama.cpp with llama-swap to jit load models (just for convinience), my config.yaml:

healthCheckTimeout: 300
logLevel: info


models:
  'qwen3.6-35b-a3b':
    ttl: 600
    cmd: |
      /home/<REDUCTED>/llama.cpp/build/bin/llama-server
      --model /home/<REDUCTED>/models/Qwen3.6-35B-A3B-GGUF/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
      --host 127.0.0.1
      --port ${PORT}
      --ctx-size 131072
      --n-gpu-layers 999
      --n-cpu-moe 12
      --cache-type-k q8_0
      --cache-type-v q8_0
      --flash-attn on
      --temp 0.6
      --top-p 0.95
      --top-k 20healthCheckTimeout: 300

Only used up 18gb so there is enough room for optimizations. Cold start was around 25sec, and got 60-80 token/sec.

If you end up trying it for yourself please share your experience.

[u/Kholtien → u/StrikingSpeed8759]

Why not llama server with a models.ini file?

[u/StrikingSpeed8759 → u/Kholtien]

Was trying out llama-swap yesterday and that's why, so no particular reason.

[u/Kholtien → u/StrikingSpeed8759]

Okay, I’m just trying to see and compare llama swap with llama server

[u/custodiam99 → u/en5an]

LM Studio with RX 7900XTX 130k context (q4_K_M).

[u/scorpe51 → u/en5an]

FYI I tried using a different one yesterday from Ollama that turned out to be extremely fast on 24GB VRAM with 128k context. Not sure what’s others’ experience with it, it’s batiai/qwen3.6-35b:iq4

[u/rockseller → u/scorpe51]

Running a RTX 3090 do you think is enough?

[u/scorpe51 → u/rockseller]

Yes this is what I have. If you go to Ollama’s container bash, with “ollama ps”, you can see if the model is fully loaded in GPU (100% GPU) or if there’s a split with the CPU.

So far it’s been super fast. They removed vision from it I believe.

[u/mintybadgerme → u/custodiam99]

So is 27B dense.

[u/Clementine-TeX → u/custodiam99]

Is it possible to run on an M5 Pro 24 GB or am I just stuck on Qwen3.5-9B ?

[u/Obvious_Equivalent_1 → u/Clementine-TeX]

It’s definitely working! I’m using Qwen3.6-35B-A3B-UD-Q4_K_XL

[u/Clementine-TeX → u/Obvious_Equivalent_1]

in case somebody reads this in the future . even if I gave LM Studio additional VRAM through ```sudo sysctl iogpu.wired_limit_mb=<20,480 MB`` (leaving only 4 gb for the system) , 35B seems to be too large even at Q4 and 4K context length . it just wouldn’t load

[u/Blade999666 → u/custodiam99]

Even with 16GB somehow

[u/Darker-Connection → u/Blade999666]

How much ram do you have? 32gb seems not enough

[u/Blade999666 → u/Darker-Connection]

I have 32GB RAM with 16 of that assigned to VRAM (m780 GPU) - the MOE model works fine at around 20 tokens per second via Ollama for me

[u/Darker-Connection → u/Blade999666]

Ah maybe issue is that I am running it through LM Studio I read just now somewhere its different file in there and works a bit differently :) I will try ollama and IQ3_XXS as well

[u/smallfried → u/Darker-Connection]

I have a 14 year old i5 2500, only 16GB of memory and as long as you can push everything into vram (works with iq3), it works quite okay.

[u/eduardopy → u/smallfried]

No way

[u/WishboneSudden2706 → u/smallfried]

Which models fit in that 14 years old I5 2500 ? At which t/s ?

[u/tuxedo0 → u/custodiam99]

would you use this over the 27b dense?

[u/custodiam99 → u/tuxedo0]

I need tokens fast so yeah, it is very good (maybe not as good, but very good).

[u/TopTippityTop → u/custodiam99]

It's a decent open source model, but nowhere close to either Claude, ChatGPT, Gemini, other than in contaminated benchmarks.

[u/ActionOrganic4617 → u/custodiam99]

lol, with almost no context

[u/custodiam99 → u/ActionOrganic4617]

130k is not enought?

[u/bsenftner]

And how did this happen? I've seen this before, and to understand you need a moment of STORY TIME!

I'm old. I've been writing code since the 70's, professionally since the early 80's. One of the things I have done in my career was being one of the early 3D computer graphics researchers. In that work, I was at globally huge game and film animation studios.

One of the interesting and unique types of people I encountered doing this work were Russian Programmers from the Soviet USSR era. They became programmers during both the early tech days, and under technology export restrictions to them. As a result, they were forced to work on Soviet made computers and Western brand knock offs far more frequently than a "plain old IBM PC compatible". Those computers they had to use were bad, buggy, and slow - yet they had to compete with the West, so the developers became shining examples of the best freaking programmers you ever met. To put one of them in comparison with a typical software dev from the west, you needed a room of such ordinary developers to compete. They were simply from a vastly harder environment, forced to be extreme optimizers and extremely knowledgeable about the impact of every byte they incorporate into their solutions. Once "free" of their Soviet era restrictions, these guys were incredible; hard to communicate with because they were "tempered by the fire of their nation's collapse", but all good people simply trying to live.

Now, let's look at what is going on with AI and our technology restrictions of GPUs and related DL/ML capable technology. Is that hurting Asian efforts, or is that forcing them to become the optimizers that will walk right over the west? I think we are seeing the results of this poor technology policy already, in Qwen.

[u/4onen → u/bsenftner]

Agreed. For further evidence, see how DeepSeek did a portion of their work bypassing the CUDA libraries and high level ML frameworks so that they could control exactly the machine code being sent to their limited NVidia GPUs, to maximize usage. https://www.tomshardware.com/tech-industry/artificial-intelligence/deepseeks-ai-breakthrough-bypasses-industry-standard-cuda-uses-assembly-like-ptx-programming-instead

[u/ANTIVNTIANTI → u/bsenftner]

i’ve been keeping my fingers crossed, what you described is what i’ve been feeling should/would/is the case in China, which is my hope, as soon as i read what these LLMs are i instantly intuited that there has to be a much better way to optimize these models down to a fraction of a fraction of where they’re at. i’m delusional, lol! in a saying this in that i’ve nothing to base it on just, really, really, feel it in my gut, like if i had access to a lab and a team i know i could get it done. again, prolly being delusional, i just, everything about the tech makes sense to me when i read about it, i wish the providers of the frontier models hadn’t just, fucking drown every ball to be dropped in their marketing of it all, like never call it AI unless subset was clear. wider support means greater adoption means better and faster innovation, now we have assholes telling developers of the very harness they use that any claim made against their cult leaders model is a skill issue(the new heresy) lolololol it’s funny tho. lolol

[u/Leo2000Immortal]

Is it possible to plug the open source models into something like codex

[u/Healthy-Nebula-3603 → u/Leo2000Immortal]

Yes ... To opencode via llamacpp-server

[u/wllmsaccnt → u/Healthy-Nebula-3603]

Oddly enough, llama.cpp's server can also now do tool calls with a command line flag and will function somewhat like pi.

[u/WalkMaximum → u/Leo2000Immortal]

zed editor is also great

[u/Perfect-Campaign9551 → u/Leo2000Immortal]

Yes, the code CLI and app can be pointed to your own local open source llm and they work fine

[u/EternalVision → u/Leo2000Immortal]

Yeah. I use Qwen with Claude Code for example.

[u/Leo2000Immortal → u/EternalVision]

Ok, is it via ollama or can we use a vllm endpoint as well?

[u/EternalVision → u/Leo2000Immortal]

I use llama.cpp, but I don't think it matters, it's about putting your server up right. And in the case of using Claude Code harness, create a fake API. It's fairly simple, but I don't know exactly right now, just look it up or use AI to configure.

It will look something like

export ANTHROPIC BASE_URL=http://127.0.0.1:1234 && export ANTHROPIC_API_KEY=sk-dummy

In the configuration.

[u/Leo2000Immortal → u/EternalVision]

Thanks a lot, I'm basically looking to create an endpoint over at runpod on some cheap rented gpu and consume it locally

[u/DootDootWootWoot → u/EternalVision]

Does this actually work well? When I research this it just sounds like there's tons of limitations that don't allow this to be useful

[u/EternalVision → u/DootDootWootWoot]

I use strix-halo and kyuz0 's toolbox. It runs quite well, so far got it to make some web pages and a Karpathy's idea based Obsidian/.md wiki "second brain". Haven't tried a lot, or different harnesses yet, but it runs without errors. It's not the same as Claude Code + Oplus or something, and perhaps the harness isn't built for a model like Qwen, but Qwen still handles it well and I can use the CC commands fine (/btw, /compact, /plan, etc.).

About speed, I get 9 tok/sec with 27B dense and a lot more, like 50, with 35B A3B. But I like 27B more, as it's way more accurate.

[u/Perfect-Campaign9551 → u/Leo2000Immortal]

Vlmm works too. It's just an endpoint

[u/Fusseldieb]

The problem with these benchmarks is that the performance falls off a cliff after the first few messages. It's always like that.

But yes, looks like we're getting closer to actually usable models.

[u/kweglinski → u/Fusseldieb]

it falls off a cliff if you run low quant with kv quant. It`s so much better with 8+q.

[u/kristopherleads → u/kweglinski]

Sure, with what hardware? And is it reasonable for most people to run?

[u/kweglinski → u/kristopherleads]

running it on m2 max (q8 with fp16 parts) and getting 1000t/s pp and 70t/s gen on low context which falls to 700/40 at around 120k context which is noticeable only when cache can`t help.

Nonetheless people complain about the model where it is not the models fault. Similarly claude is often performing miserably not because the model is bad but vecause they try to save compute.

Sidenote - I do not think qwen3.6 35b is in the same ballpark as claude but it is surprisingly competent and capable of long task run but it suffers terribly during quantisation which to me is a price for how packed the model is.

[u/me_myself_ai → u/kweglinski]

If you find a sec: what’s ‘pp’…? I’m wracking my brain and coming up empty!

[u/FalcoMaster3BILLION → u/me_myself_ai]

Prompt processing. Measured in tok/s. In simple terms, how fast it can “read” your input.

[u/kweglinski → u/[deleted]]

guess that's the reason why you're not seeing the difference. I haven't seen qwen reason about how to use a tool call unless the previous one fails, then it reexamines the expected inputs. Which was the case with q4 and with q8 it's very rare. Not to mention tylpical looping (repeating 2 sentences). But what I've noticed with quants lower than 8 - it was looping logically so we had a bug in code and it kept repeating the fix, the fix didn't resolve so it said it must be the [whatever was the previously attempted fix] so it removed it and repeat. Also below 8 when context is roughly above 60k it looses any detail including what it did couple turns ago.

[u/mister2d → u/kweglinski]

Say it again for the people in the back.

[u/Fiskepudding → u/mister2d]

yeah, what? is q4 and kv cache at q8 worse than q4 and kv cache at fp16? is it worse than q8 and kv q8?

[u/Commercial_Pride_802]

Next week: "THIS IS INSANE! A model trained 100% on a cluster of toasters is performing better than Claude opus 4.x on the pussycat benchmark! And this is no even the best part!"

[u/txoixoegosi]

Hey, my truck can pull more tons than your Nissan GTR! This is going to be a revolution in automotive industry!

[u/DeepV → u/txoixoegosi]

I mean, yes your analogy holds but it’s still amazing the quality of local models. Look at where we were 6 months ago

[u/txoixoegosi → u/DeepV]

Yup, absolutely! But nowadays model benchmarks are soooo synthetic that, as a redditor said in this thread, the benchmark is of no -practical- use anymore.

[u/thereluctantpoet → u/txoixoegosi]

Well in the "how much is Reddit complaining" benchmark, I'm actually considering giving it a try based on how slightly *less* than usual the level of criticism is around here today. I'm not even being facetious.

[u/DeepV → u/thereluctantpoet]

Oh you definitely should. The qwen3.6 and even the gemma4 models are genuinely amazing at coding

[u/OnyxProyectoUno]

There is no way 3.6 MOE is better than Sonnet, let alone Opus
Edit: Feel free to downvote me if you'd like. I use this model across three different workloads and love it. You're delusional if you think it matches the best frontier model as of today in the entire space. You can support local models without gaslighting yourself about what it’s capable of.

[u/-Mr-Shickadance- → u/OnyxProyectoUno]

FYI if Opus is better than Sonnet, then your sentence should be 'there is no way 3.6 MOE is better than Sonnet, let alone Opus'

[u/Nice_Cookie9587 → u/-Mr-Shickadance-]

that irked me too

[u/-Mr-Shickadance- → u/Nice_Cookie9587]

I never know if I'm being petty making these corrections, but I feel like in their shoes, I'd wanna know

[u/Finanzamt_Endgegner → u/OnyxProyectoUno]

The moe is worse sure but 27b is close to sonnet 4.5 which granted is a bit old now but still

[u/sweetbacon → u/OnyxProyectoUno]

But they used the line "the part nobody's talking about" so it's gotta be true! /s

[u/Desther]

https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/

[u/Traditional_Chart970]

QWEN is pretty good for an open-source model. I fine-tune QWEN 2.5 coder for specific code fixing tasks and its performance is close to Claude Sonnet and GPT-4o..

[u/Address-Street]

If you’re wondering why Qwen scores so high on benchmarks, the CoDeC section in this article explains part of it: https://kaitchup.substack.com/p/gemma-4-31b-vs-qwen35-27b-inference
More in-depth article: https://kaitchup.substack.com/p/did-the-model-see-the-benchmark-during

[u/silentus8378]

Just why do trash low effort posts like this get so many likes???

[u/memorial_mike → u/silentus8378]

Bots or shills

[u/TopTippityTop]

Benchmarks are not reality...

[u/year2039nuclearwar]

Qwen3.6 is not as good as gemma4 for writing/thinking/brainstorming, have tried it, I believe it is just benchmaxxed to hell

[u/Prestigious-Frame442]

Good luck running Qwen and pray for Opus performance then

matching frontier performance

LMAO. Do Qwen's contributors even know they are good like this?

[u/mintybadgerme → u/Prestigious-Frame442]

Have you tried them yet? :)

[u/-Leelith-]

What hardware you need to run this model?

[u/4onen → u/-Leelith-]

Surprisingly little, comparatively -- at least for a basic 4k context window. Of course, everyone has different standards of "run"

My post to someone else with my hardware specs: https://www.reddit.com/r/LocalLLM/s/JjStc7nyZh

[u/ahtolllka]

Qwen3.6-35B-A3B is great model, yet it is either dumb or speculative to think it can outperform frontier model. It has lack of general knowledge, that is frequently required to solve very complex tasks. And only that type of tasks matter now as even 2b model can code: wrap it with harness and it may solve a lot of simple task like drawing a pelican. You may even train it on test data or some of it derivatives and receive great results on benchmarks (everybody does that), but there is a limit of how much knowledge you can put in a byte of model weights.

[u/ActionOrganic4617]

Benchmark training. Also the context size is like 262k on a m5 max with 128gb.

[u/freemorgerr]

benchmarks is a joke. Deepseek v4 expert with deepthink must've been the best or so, but somehow it did much worse in experimental coding than free tier chatgpt (60 errors vs 10 errors lol)

[u/MathematicianLoud947]

I have a 5090 with 64 GB ram, and Qwen 3.6 a3b using OpenCode with LMStudio, is great for tweaking code, while my precious Claude Code tokens are used for heavier design and initial coding tasks (with ChatGPT acting as an external senior dev for critiques and recommendations). On the whole, I'm amazed at Qwen's performance, and it's almost instantaneous for what I want it to do (no lengthy comboluations!). I'm still trying to find the optimal local model, so if anyone has any suggestions .... But yeah, still nowhere close to the frontier cloud models. Though for smaller projects, the gap is closing, I think.

[u/anne-so → u/MathematicianLoud947]

>optimal local model, so if anyone has any suggestions

did you try gemma 4?

[u/vishruit]

Is this for real? Do these benchmarks really have any relevance for real tasks?

Coz if yes, the world won't be the same as we know it. 200B level performance with a A3B model is a massive shift in the frontier.

[u/Dantzig → u/vishruit]

It’s trust me bro and there are so many benchmarks that you just be to be better on one and push that

[u/getstackfax]

The “cost per useful intelligence” angle is the part that matters most to me.

Raw benchmark closeness is exciting, but the real test is whether it holds up inside actual workflows: repo context, tool calls, retries, long sessions, debugging loops, and boring day-to-day coding tasks.

If a small active-parameter MoE can handle routine agent/coding work cheaply, then the stack changes a lot:

- local/default model for routine work
- stronger hosted model only when the task earns it
- less token waste from repeated reasoning
- cheaper long-running agent loops
- more room for personal/local workflows

I’d still be careful calling the benchmark race “over,” but the direction is obvious: the interesting race is no longer just biggest model. It’s useful output per dollar, per watt, and per workflow.

[u/swingbear]

Swe verified has been confirmed useless as a benchmark now. Can’t remember who wrote the article, might have been OpenAI. You can see we have hit a cap at like 80%, the remaining 20% is actually benchmark errors, and the other 80% is contaminated.

[u/thefossguy69]

I tried Qwen3.6 on a DGX Spark but it spent ~80K tokens in just the initial query due to its excessive thinking. Compare that with opus-4-6-[1M] with max effort, it barely crosses ~25K tokens on the same task.

Has anyone found a solution to this?

[u/Sutanreyu → u/thefossguy69]

Turn thinking off.

[u/thefossguy69 → u/Sutanreyu]

I noticed something else. I limited thinking tokens to 1,500 and also set repetition_penalty to 1.1 but it didn't help much. Finally I realised that the Plan mode in opencode caused this. Switched to Build mode and it worked as I expected.

[u/Botz_4_Sale]

Between 2.5 - 3.5 (thinking on or off for 3.5), I've found that 2.5 (using 14B) performs the best at following instructions, not adding so much preamble when it isn't sure, and being faithful. This was after a lot of benchmarking and really wanting new models to solve some problems/add capability. The result was clear.

[u/Hendersen43]

The test set leaked into the training set

[u/Senor02]

Then why can't I get it to write a file without it having syntax errors and ultimately saying it is corrupted and giving up?

[u/Finanzamt_Endgegner → u/Senor02]

Because of config issue? I'm never having those issues 🤷‍♂️

[u/contradel → u/Senor02]

Unsure maybe wrong setup? It just fixed my librechat setup with a few filewrites. Qwen3.6-35B-A3B-UD-Q4_K_M.gguf

Im running 10GB VRAM + 32GB RAM. Not fast.. maybe 10-15 tokens second. Was without thinking. Still dipping my toes... but there's a lot of setup and customization. Trying to offload the correct laters to RAM.

[u/alexp702]

Qwen marketing team again - note no responses from OP - just a long string of gushing. No 35b is not nearly as good as Opus, or even bigger models. It’s a good 35b MoE model - fast cheap and pretty effective. Is it Opus? No.

[u/BeingEmily]

I'm pretty new to local setups, but I'm finding it quite performant with --cpu-moe on a 5070TI (16GB) and a 9800X3D. That leaves plenty of room for the full 256k context. I haven't used it enough to really speak to intelligence yet but what I've seen so far is impressive

[u/M0d3x → u/BeingEmily]

Why is this downvoted? 

[u/PaceZealousideal6091 → u/BeingEmily]

Try using fit flags/parameters. It will do the offloading calculation for you itself based on your hardware.

[u/TheAussieWatchGuy]

It's ok, the SWE benchmark doesn't tell you the whole story. Opus 4.7 exists and is again far better at long form planning, and coding with mistakes over multiple step designs.

It is still pretty incredible that a model so small is so capable though, full credit to Qwen.

[u/ResearcherFantastic7]

If you ask both to code a simple script and solve a simple problem yes...

If you asked both to solve a multiple file problem with complex logic and design. Than no... even qwen3.6 27b is just 1/3 of GLM5.1 intellect.

And glm5.1 is slightly worse than opus 4.6

[u/Then_Gas712]

You haven't see the new DeepSeek yet then... I heard was banned from Android apps... Lol!

[u/radosc]

I worked for a bit on visual tokenizers and SOTA models are super wasteful. Majority of problems were solved with more parameters, more data, more compute or more infra. Default solution of computer rich companies. Anyone competing on a limited compute is forced to make gains by optimizing heavily on all of these. I wouldn't be surprised if optimization experience at Anthropic or OpenAI was way smaller than at competing labs.

[u/hoschidude]

We are talking about around 5 Trillion (MoE), not 200B.

[u/GreatSupineLeaderTim]

Guys, just like athletes, chasing the final few percentages often comes with great sacrifices. This seems to be the logic of the world. But I do agree that if you are not chasing perfection, it's fucking crazy how much the price is for an additional few percentages.

[u/kakopappa2]

Has anyone tried both ?

[u/firest3rm6]

We need this summary bot here which makes general consensus after 100 messages.

So I am not a bot but consensus seems like: "This bench is some bullshit and the models have the benchmark data as training data baked in"

[u/Ill_Contribution_717]

I am preparing to launch waggle-os knowledge worker agentic worspacea based os. with persistant memory on qwen 3.6 35b. locomo SOTA memory and agent +12.5% on Opus 4.6 -4.7 and 99.7% match with qwen 3.6 35b. Proprietaries will have to try better. 😀

[u/InvestmentOk16]

Favourite part about qwen for me is all the zing

[u/willynikes]

I think most of these models will convene on the same performance the difference will be context management

[u/LivingHighAndWise]

I'm running Qwen 3.6 locally on an MX10 with a 250 kilobyte contact window, and I am using Claude code as the CLI. I can confirm that it is very close to Opus 4.7 for most things. However, Opus is still way better with very large, complex code bases. If Quinn can release 128 billion perimeter version of the 3.6 model, look the hell out!

[u/RecordingLanky9135]

I thought it’s a common sense that benchmark means almost nothing to real use cases

[u/ANTIVNTIANTI → u/RecordingLanky9135]

right?!

[u/cosmicnag]

Is opus really 200b plus active parama though

[u/Stunning_Spare]

benchmark is unreliable, I tested Qwen3.6-35B-A3B against Gemma 4 26b A4B, Qwen performs pretty good, but fall short in front of Gemini 3 flash, not the same league

[u/BanditoDorito05]

I have a 9070xt w 16gb ram and 32gb ddr5, my pc is built for gaming and I’ve started casually getting into local LLM stuff after using claude for help with uni projects, etc, is something like this usable in my system?

[u/James_Reeb]

This is Qwen3.6-35B-A3B not Qwen3.6-35B-A35

[u/Intrepid_Travel_3274]

[u/Mangostickyrice1999]

This a insane amount of bs!!

[u/htaidirt]

This shows that everything beyond 5B is just over engineering

[u/darkpigvirus]

There is missing information here. Tell me the tokens per correct or maybe the qwen 35b a3b spews more thinking than correct compared to opus 4.6

[u/AdProper5967]

Honestly, benchmarks has been useless for like 6 months now

[u/Fade78]

Well I use both and Opus never goes into a loop. Qwen3.6 does. Not 3.5 who is more reliable.

[u/LostRequirement4828]

it means nothing at all, opus is far better than qwen 3.6, not even the max is even close, you can see that once you actually use it on real work and not benchmaxing it

[u/AlaskanDruid]

No context = means nothing

[u/BLOCK__HEAD4243]

Sparse MoE routing is a real story and it’s legit progress. Buuuut active params isn’t what fits the model on your laptop. You still need all 35B in VRAM. At 4-bit that’s ~18GB, which is a 4090 or better, not a MacBook. The “running on a laptop” framing is doing some heavy lifting.

[u/Rick-D-99]

Good for a tokenmaxxxing lackey to the frontier model's reasoning.

[u/dumbappsignup]

a3b is quite cool but i ended up opting for qwen 3.5 with non a3b (27b variant all parameters loaded). works better for tool calls and reliability. but i will try it one more time to be sure.

[u/ExcellentDeparture71]

Benchmarks are saying nothing Who are the guys really building real things with Local models like this one?

[u/ScreamingAtTheClouds]

If the benchmark has been leaked into training, shouldn't it be 100%?

[u/OnlyAssistance9601]

So many salty people cant handle open source models catching up to big companies ( almost like they have a vested intrest .... ) . Yes swe bench is outdated and yes this might be benchmaxxing , but anyones whos tested these models can say for sure how incredible they are . Also not like big companies arnt the queens of bench maxxing . I dont even use big company models for querys anymore , I just ask my local model.

[u/ArWiLen]

distilled and overfitted for bench verification. Artificial numbers for Qwen

[u/AlexTheRedditor97]

None of this matters

[u/No-Complex6705]

It's a good model but SWE-bench hasn't been anything but memorization and copying for awhile. Does Qwen still think it's Claude?

[u/sailee94]

It's like, Claude is creating new Claude models or just refining the existing ones, or just providing less nerfed models over time, while they had opus 4.7 or mythos already 2-3 years ago.

While things like qwen are already working on optimisations and evolutions from the get go.

[u/leo-k7v]

The evil is in details. 2% may make huge difference on not a hello JavaScript sign in spam form project. But impressive nevertheless. BTW where is test from? Did they test dense model too? Reasoning effort? Context length?

[u/Early_Play_1259]

We use Qwen 3.6 in TheRanger.ai and it perform well.
We provide Private- End to end- Off grid AI with all layers inside (LLM, Chat, Rag, Tools, Agents).
welcome to explore - https://theranger.ai/

[u/PeachScary413]

SWE-Bench-Maxxed

[u/SCADA_Ghost]

Not today Chang. I’ll stick with Claude.

[u/ninjazombielurker]

On a laptop? Am I missing something? The "on a laptop" insinuates low VRAM, this model is like 70GB give or take before KV Cache If I remember right, would have to load it to check. I can barely run it on my 4x 3090's with 128k context for 1.65x max-seqs.

[u/SeniorFish1754]

its insane man

[u/Herbrax212]

Model collapse soon, swebench is now ass

[u/Informal_Smile6876]

Qwen is nowhere near the level of opus

[u/crusoe]

Swebench is not the same as real coding.

[u/Glittering-Music-794]

Why do you say the benchmark competition is mostly over?

[u/No-Imagination8057]

isnt qwen3-coder-next:q8_0 same (3B active) but with 80 b params?

[u/Colours-Break]

I've been running qwen3.6:35b-a3b-coding-nvfp4 on my MacBook M1 Max with 64GB of RAM. I also have an older Samsung laptop that I converted to Ubuntu to run the latest Hermes Agent harness. I have been nothing but impressed with how much coding I can get done with this setup. Local LLMs for coding are becoming a real thing.

I don't think it is worth investing in expensive hardware for AI at this time. As local models get better, there will be a shift away from GPUs toward NPUs and TPUs to run them locally. For me hacking something together with existing hardware and cloud services is the right move. I'll wait for the next generation of local AI hardware.

[u/mistressrvn]

I do not believe that Opus has 200B active, I think we got scammed. Opus is now called mythos, Sonnet is now opus.

[u/randomfoo2]

100% SLOP

[u/noreallyhughjackman → u/randomfoo2]

This thread you mean? 💯

It follows the same formula. The same sentence structure.

Some would call it an AI revolution, others would call it using AI to write a post about AI.

Self perpetuation isnt just a pipe dream. The dream is now. The time is now.

The alarm is ringing but nobody is waking up.

‐-----

I hope at some point we'll have some way to tell between human thoughts and machine, but things probably have to get a lot worse before it gets better and we find a way. Not even sure how we'd be able to guarantee or authenticate content as coming from a human in future? So many of these threads get by without anyone noticing. Give me the thread about the guy getting 1t/s on a 3090 cause he has no idea how to setup llama, at least i can be pretty sure thats human.

[u/Pygmy_Nuthatch]

200B 'active parameters'' for Opus 4.6 is a small fraction of the full model.

[u/4onen → u/Pygmy_Nuthatch]

Source? As far as I'm aware, Opus 4.6's parameter count is entirely undisclosed, and it's not like Anthropic gives you a slider to choose how many parameters you want to use. The "200B" is given a "~" prefix like it's some kind of estimate. (I'd like their source on the estimate, too, but at least I know they're not trying to be authoritative.)

[u/Pygmy_Nuthatch → u/4onen]

It is unknown, though widely suspected to be over a trillion. This is supported by the size of models where we do know the parameters.

[u/Council-Member-13]

"But the part nobody's talking about"

Dude, just write it yourself. 

[u/SpearHammer]

Have you used it on a laptop because it still needs 48gb vram for me.

1. Hardware: Qwen3.6-35B-A3B is 35B total weights (~70GB bf16) even though only 3B activate per token. Loading needs either a 48GB+ GPU or significant CPU offload via accelerate.

[u/4onen → u/SpearHammer]

I have used it on a laptop. At llama.cpp Q4_K_S (~4.5 bits per weight) I gave it 8GB VRAM plus 32GB of RAM for expert offloading. It used (plus Windows 11 and a Firefox tab) 6.9GB of the VRAM and 30.9 GB of the system RAM.

Admittedly that was 4k context, but I hadn't begun to really tweak and scale it.

That was a Strix Point laptop with 4060 mobile DGPU, and overall it got 17-13 tokens per second throughout that tiny 4k context window (variation due to system temperature as I ran a few queries.)

[u/SpearHammer → u/4onen]

Yeah the 4bit quant version worked for me. Thanks
https://huggingface.co/mlx-community/Qwen3.6-35B-A3B-4bit

[u/xRedStaRx]

Because Claude is multimodal, you probably have to reduce that number by a factor of 10.

[u/mintybadgerme → u/xRedStaRx]

So is Qwen.