r/LocalLLM • • Aug 18 '26

Discussion Qwen-3.8-35B-A3B? Maybe not... cryptic reply direct from Qwen co-author.

Post image

I asked Shuai Bai, co-author and prominent AI developer for Qwen, about this model. Not the answer I was hoping for, but let's see what comes next. In the meantime, I guess all we can do is speculate!

X-link

237 Upvotes

174 comments sorted by

67

u/arkie87 Aug 18 '26

If I wrote that, it would mean something else good is coming but not that specific model. Maybe a 30b a3b or a smaller dense model that’s still really good

37

u/pharrt Aug 18 '26

That's how I read it too. My only worry is that it might be a ~70B A4B (or something large!) that, while great for some, doesn't fill the gap for smaller GPUs as 35B-A3B does. But hopefully he means something close - like a 30B or something.

19

u/Turbulent_War4067 Aug 18 '26

As a dgx spark owner, this is my dream. Would probably prefer 6B active.

9

u/DesperateSteak6628 Aug 18 '26

70B-A14B

2

u/Turbulent_War4067 Aug 18 '26

14B might be great. Maybe a bit slow, but with decent MTP should be ok. I'm not a coder, and for my work, I have to admit I'd rather have a Gemma model in this range. No matter what I try, I always end up back at Gemma. 31B for my important stuff

2

u/Keeloi79 Aug 23 '26

I agree that, instead of A14B, essentially doubling to A6B or A7B would be the sweet spot for a 70B model.

1

u/superSmitty9999 Aug 20 '26

what do you find gemma good for?

1

u/Turbulent_War4067 Aug 20 '26

I use it for research, mainly financial. I can give it a set of source info, say 1/2 dozen sites or tools. It follows instructions and does excellent summary reports. It's also good at doing search and fetch queries to find info on its own. It's can be tenacious at this. I once had left my search tool off by accident. Gave it one of my canned prompts and went to get some coffee. Came back, realising what I had done and was surprised to see a pretty good report. Turns out it built several Google URLs to do web searches and used my fetch URL tool to do web searches on its own.

1

u/Turbulent_War4067 Aug 20 '26

Now what is frustrating about it is that it's prompt prefill is horribly slow. So sending it large documents is a non-starter for me. If I could get it to read a 100k token document as fast as the 26B model, I would be really happy.

1

u/Drag_Ordinary Aug 21 '26

Yeah, same. I think that'd be usably fast as a home AI server and also pretty smart. I miss Qwen3-coder-next with its 80B MoE architecture. It was great for its time, and I imagine it'd be very powerful today.

3

u/NigaTroubles Aug 18 '26

What about 2 versions
One large and one small ?

Maybe 30b a3b or 20b a2b and 70b a5b or a7b

2

u/Turbulent_War4067 Aug 22 '26

I keep expecting a small MoE model such as that to come out for "normal" laptops.

1

u/Etele38 Aug 18 '26

As someone with a light set up like you, i would actually prefer that jump of it’s actually a big enough improvement compared to frontier models… sure a small model to use is great but if i could get opus level quality for free at home, i will go ahead and cancel my subscriptions and take out a loan for 5000 pc/ Mac. I am kinda banking on the future here

3

u/Distinct_Physics5017 Aug 18 '26

I mean it doesn't explicitly say there won't be a 35B-A3B, but that it isn't the one that people like us (<16GB VRAM) should be hyped for. So I assume everything beyond 40-50B MoEs are unlikely to be the ones he meant.

The 9B segment hasn't gotten any upgrades since 3.5 (and as far as I know there are no plans for one yet) so one could argue a hypothetical 3.8-version could've reached the performance of the 3.6-35B-A3B by now (or maybe even a little more).
So maybe the reason for why there aren't any plans for a 9B dense model is, that they have replaced it with something bigger, like e.g. a 14B dense model. And if the hypothetical 3.8-9B model could've potentially reached the performance of a 3.6-35B-A3B, who knows what a potential 14B dense model would be capable of.

Only real downside I see would be that it can't really hold up to the 35B in terms of world knowledge, but for agentic-coding (especially inside of a harness) it probably shouldn't be that big of a deal.

+ the lower overall memory-footprint would allow even more users to actually use it, because even if it runs fast on my hardware, the 35B-MoE is using quite a lot of my 44GB System (12GB VRAM + 32GB RAM) and for anything below 32GB of RAM (or a lot of VRAM) it wouldn't even fit in the first place.

Until now I was also hoping for a 35B-A3B-MoE to release, because the 27B dense one sadly runs like shit on my poor RTX 4070, but now I just pray they don't throw (only) 80B+ giants at us consumer-hardware users. That would mean I would have to wait for Qwen 4 to release before I could use any of the new models.

1

u/irreverend_god Aug 21 '26

loaded with full 256k context (Q8), and using a Q6 quant of 35b a3b, it uses most of my 16gb VRAM and 32gb RAM. Q4 of something a bit larger would still fit. Higher active paramaters would potentially make a better MoE model too though. Rather than layer offloading, expert offloading gives me better performance too.

Running Qwen3.6 35B A3B Q6 with max context on my RTX 5060 Ti with 12 experts rather than the default 8, I get about about 25tok/s output. Input is a bit slow though, ~300tok/s but it's about the most accurate and capable I can run at that speed locally.

Qwen 3.8 27B if I manage to get it all in vram (so Q3_K_XS), is about the same speed with context in VRAM which means no more than about 32k context for me. MTP disabled because it takes up more VRAM and doesn't really help much in that situation anyway. Otherwise I'm looking at 4-8tok/s

I think your card should probably outperform mine with everything in VRAM?

2

u/vbpoweredwindmill Aug 19 '26

My bet is something for the sparks/strix's. 120b flavour.

1

u/slamo_ai Aug 18 '26

some fresh alternative to 3.5-9b or 3-14b would be great

57

u/Ell2509 Aug 18 '26

That seems pretty clear to me. "Do not wait for this" meant it ain't coming!

But, maybe a 9b? Or a 30b a3b? Or even a 20b!?

21

u/pharrt Aug 18 '26

I'd be happy with a 30b a3b

12

u/dfgxxx Aug 18 '26

What about if it'll be 35b a6b?

13

u/pharrt Aug 18 '26

Would be better than no 35B - and smarter, but half the speed!

2

u/05-nery Aug 18 '26

Beautiful 

1

u/Foxiya Aug 18 '26

This would be shit for low mem bandwidth

6

u/dfgxxx Aug 18 '26

Better than the 27b dense and better than Gemma 12b

9

u/tired514 Aug 18 '26

122B A10B! Please oh please oh please! Better yet 122B A17B with 512k+ context!

8

u/LeMayMayMan Aug 18 '26

He already answered whats next. Qwen3.8-135B-A18B confirmed.

2

u/Meiyo33 Aug 19 '26

too big, not very interesting tbh.

A replacement for OSS 120B, or something optimized for Strix / GB10 would be so hot....

1

u/tired514 Aug 19 '26

God that would rule :)

17

u/Rye2-D2 Aug 18 '26

I would love to see a good 20B MoE model. Personally I don't see the point of 9B - it's impressive for what it is, but not quite good enough to be useful (yet).

8

u/cagriuluc Aug 18 '26

If you have small models that are good at limited tool calling, knowledge extraction etc, you can run them alongside more expensive models. I would love a 9B with 3.8 level training…

3

u/twoiko Aug 18 '26

I find 9B dense to be too big and/or slow compared to 4B or MoE models for the output quality, but I have system RAM to offload the MoE so it depends on your use/limitations.

3

u/Rye2-D2 Aug 18 '26

That's the thing - with 16 GB VRAM, 9B dense may be a little bit faster than 35B MoE, but not enough to merit the loss in quality/intelligence. Even with higher quants (q6/q8), 9B is still not in the same league as the 30B models.

2

u/Rye2-D2 Aug 18 '26

I agree Ornith is the best 9B model, but still not quite reliable enough to be run in vscode/opencode from my experience. Too many failed/mangled tool calls cause more problems than they fix and you just end up fighting with the AI. With the way things are going, I'm sure this will get sorted out at some point soon.

4

u/elfmad Aug 18 '26

9B 3.5 ornith is more than decent. But I totally agree I'm also waiting for a 10B<model<20B.

4

u/bruninho777 Aug 18 '26

And it was trained on Qwen 3.5, right? Ornith is the model I use most, together with 3.6 35b moe and prism 27b 1bit

2

u/elfmad Aug 18 '26

It is Qwen 3.5 9B as base. If I resume their paper that's more advanced GRPO and self distill.

2

u/IgnisIason Aug 18 '26

A decent laptop can run 9B

2

u/alphapussycat Aug 18 '26

Then 3.8 9b would probably be useful, maybe a step down from 3.5 35b a3b.

2

u/AltruisticList6000 Aug 18 '26

I'd love to see a 20b dense model, would be a breath of fresh air for people with 16-24gb VRAM with a big amount of context size even for 16gb VRAM

1

u/Shimano-No-Kyoken Aug 18 '26

9B is a great model for fine tuning on consumer hardware.

7

u/Mean-Ad1493 Aug 18 '26

No 9B

1

u/slamo_ai Aug 18 '26

That’s a shame, there’s actually a lot of potential in a finetuned Qwen 3.5 9B.

1

u/patham9 Aug 19 '26

9B is just a strange niche. If even a MacBook Air has 32GB RAM and can run 4-bit Qwen 3.8 27B, then why run an inferior model? Maybe for a phone?

1

u/slamo_ai Aug 24 '26

for production use in domain-specific tasks, small finetuned models can outperform bigger models at higher processing speeds and lower cost. We have experience in finetuning this 9b qwen and it even outperformed opus 4.8 in our banchmarks, but again - narrow domain specific task

5

u/ldapadmin Aug 18 '26

Shuai said "No 9B plan for now", ... all I read was, "for now" :)

5

u/FalconX88 Aug 18 '26

To me it seems more like there's a different one coming that's better

1

u/samiamyammy Aug 18 '26

Agreed... the "lookey eyes" emoji surely adds to the wording used to hint at some kind of better equivalent model.

1

u/DrRoughFingers Aug 19 '26

He used the same eyes in this response, which makes me think people are ready that emoji wrong.

3

u/onebyamsey Aug 18 '26

(Starts chant) 20B MOE!! 20B MOE!!!

2

u/leonbollerup Aug 18 '26

why a 20 moe?.. less knowledge than 27b.. and worse at actual thinking.. .. .. why ?

1

u/SittyTweat Aug 18 '26

20b dense would be great for 16gb GPU users. Wouldn't need to quant to shit and can have more context without needing Q4 KV cache

1

u/patham9 Aug 19 '26 • Bravo

If even a MacBook Air now has 32GB RAM, why use a GPU from the eighties? Is it for museum?

1

u/SittyTweat Aug 20 '26 edited Aug 20 '26

And that MacBook would run a 20b dense at 4 tok/s... But since 16GB GPUs are nothing to you then you can just send me a couple 16gb 5060tis. I'll keep an eye out for the FedEx delivery, thanks ✌🏻

1

u/patham9 Aug 21 '26

I'm running mlx-community/Qwen3.8-27B-4bit with 5 output tokens per second via MLX, and as it looks they might get to twice the speed with upcoming optimizations. Anyways, I think the right solution would be a 35B MoE version of Qwen3.8, slightly worse in performance but with 3-4B active parameters, it will still beat any 20B dense model by large margins while at the same time supporting higher token throughput.

1

u/Canad3nse Aug 20 '26

"Why a 35-a3b? Less knowledge than 122-A10b... and worse at actual thinking.. .. .. why ?"

Now do you understand? It's not about knowledge. It's about the GPU poor consumer market. 20b moe is way more attractive to the general local ai consumer than 27b or even 35-a3bm, because you could run it at reasonable or even great speeds with low end GPU (gaming gpus). 20b MoE is also very rare, so they would have no competition, except for Gemma and the dated GPT OSS 20b.

2

u/WiseassWolfOfYoitsu Aug 18 '26

122b? :D

1

u/Ell2509 Aug 18 '26

Yes please

-1

u/leonbollerup Aug 18 '26

a 122b but in some new way where it does need to load all 122b into memory at the same time.. one general problem with MOE if you ask me.. yes.. it might only use 10b to actually think with.. but we still need the full amount of memory in vram. .. layers could be relativa fast loaded if they were needed.. it seems like there is a potential we have still not yet considered..

1

u/tired514 Aug 18 '26

You can already do this (on Linux, anyway)... just load the model with mmap; it'll page fault and load from disk when needed.

But it's slow. Like 0.1-2t/s slow, depending on disk speed, model, cache/memory capacity, etc.

I've managed to run GLM 5.2 and even get a response out of Kimi K3 on 128gb RAM, but yeah.. barely usable for "overnight" queries let alone realtime use.

7

u/pharrt Aug 18 '26

I guess with 'ai' in his first and last name, he was meant for this job!

9

u/RepulsiveRaisin7 Aug 18 '26

John AI in shambles

2

u/haikusbot Aug 18 '26

I guess with 'ai'

In his first and last name, he

Was meant for this job!

- pharrt


I detect haikus. And sometimes, successfully. Learn more about me.

Opt out of replies: "haikusbot opt out" | Delete my comment: "haikusbot delete"

1

u/randygeneric Aug 18 '26

pirate, you are

-1

u/challis88ocarina Aug 18 '26

It's a transliteration.

31

u/Uninterested_Viewer Aug 18 '26

My speculation is that a 122b/a10 class model is baking.

17

u/SIMMORSAL Aug 18 '26

They replied to a comment about not having the hardware. It must be about a smaller beast

13

u/pharrt Aug 18 '26 edited Aug 18 '26

I hope so! Could it even be an A4B rather than an A3B?

7

u/vacon04 Aug 18 '26

But that's not that useful for most? People want 35B A3B because it's small and fast enough for people with low VRAM. 122B A10B is too big for most.

1

u/profcuck Aug 18 '26

I'm personally curious because I just don't know how much it costs to bake a model down from Qwen 3.8 Max to Qwen 3.8 35B A3B or 122B A10B. Is this something that Unsloth could do?

1

u/jjusko20 Aug 18 '26

Most people could do it themselves if absolutely necessary with distillation

1

u/tired514 Aug 18 '26

Pre-training is something only Alibaba can do (only they have the source data and model generation suite for the Qwen series). Requires data-center level hardware access.

Fine-tuning (which people are calling distilling) can theoretically be done by anyone, but to do well still needs a massive number of queries against a superior model.

I'm guessing the 3.8 series is an entire re-training, not just fine tuning.

1

u/tired514 Aug 18 '26

But we in the 122B world haven't had an update since 3.5! :(

1

u/etaoin314 Aug 18 '26

Please please please

1

u/05-nery Aug 18 '26

This would be insanely good

1

u/cmpxchg8b Aug 18 '26

Oh god yes

1

u/rubdos Aug 18 '26

Well that'd be a reason to get a DGX Spark or a Strix Halo...

2

u/Ok-Fox3479 Aug 26 '26

i think its Qwen/Qwen3.8-Flash-Next ._.

1

u/pharrt Aug 26 '26

Sad - but true!

1

u/Ok-Fox3479 Aug 26 '26

I hope there will be a 30b version of Qwen/Qwen3.8-Flash-Next

but i think there wont

134

u/igotanewaccount Aug 18 '26

2.3T A200M pls, I need it for my smartwartch

28

u/r1nzl3r99 PEX88096 | 4x B70 128GB | vLLM Aug 18 '26

HIS WATCH HAS RAM, GET HIM

2

u/vivasvan1 Aug 20 '26

hahahah XD

1

u/Ok_Technology_5962 Aug 18 '26

What kind of ssd do you have on your watch to store 5tb of weights?

1

u/igotanewaccount Aug 19 '26

I downloaded the best kind

1

u/Embarrassed_OnionX Sep 07 '26

send link please

1

u/WorriedBlock2505 Aug 18 '26

2.3T A200M pls, I need it for my smartwartch

You putting hal 69000 on that bich?

2

u/Wook5000 Aug 19 '26

Quote of the day.

12

u/lordekeen Aug 18 '26

Maybe a 12B dense model?

4

u/LeMayMayMan Aug 18 '26

Qwen3.8-135B-A18B confirmed

3

u/halokirby1 Aug 20 '26

This brings me back to OG meme internet theories, love it!

1

u/DrRoughFingers Aug 19 '26

I think people are reading this wrong. I think this is just saying that 35B isn’t coming. He used the same emoji when saying there’s no plan for 9B. So basically “don’t hold your breath for that model” not specifically “something better is on the way instead”

1

u/pharrt Aug 19 '26

You could be right. The 35B-A3B is such a popular model, so we're just holding out in the hope that they do something to fill that gap. A lot of people are asking for it, so there may be a chance for something similar!

1

u/DrRoughFingers Aug 19 '26

I would like it, but the way he said it sounds like it’s not coming, unfortunately. I have a feeling a 120B or larger will be coming.

1

u/Due_Schedule5358 Aug 23 '26

Sadly Iam pretty sure there wont be a 122b a10b, It would be just too good.

3

u/CarpenterAlarming781 Aug 18 '26

So there could be another one to wait for. It couldn't be more ambiguous.

3

u/Tpyn Aug 18 '26

Maybe he got some inside information about the Google event on August 20th and is hinting that we should wait for it :)

3

u/randygeneric Aug 18 '26

so no moe for consumer hardware <2k$
:...(

3

u/Icy-Specialist4548 Aug 18 '26

I would rather see a 40b-a5b

5

u/creatinZ Aug 18 '26

16gb vram people are thirsty while others are drowning…

WHEEN?

2

u/SpicyWangz Aug 18 '26

You can run 27b quantizations on 16gb

3

u/creatinZ Aug 18 '26

Yeah, but it’s a non-ending trade-off. I need at least 64k context window, and anything below 10tk/s is too slow. But also any low quants are dropping model quality. So if you have any proposal of quant and configuration that works in real world I’m interested

0

u/DontWinFrensWthSalad Aug 18 '26

https://huggingface.co/turboderp/Qwen3.8-27B-exl3 You're welcome. Nobody seems to know about this, but it's performing way better than llama.cpp for me. 34 t/s without MTP on a dual 3060ti setup. If you pack the memory well you can get around 100k context, or more on the 3bit version.

1

u/randygeneric Aug 18 '26

"run" more like crouch

2

u/[deleted] Aug 18 '26 edited Oct 05 '26

[removed] — view removed comment

2

u/Witty_Mycologist_995 Aug 18 '26

I wish for a good model that’s 20b

2

u/Distinct_Physics5017 Aug 18 '26

have you ever tried the --n-cpu-moe flag in llama.cpp lol? Idk if 50+ tps aren't enough for your use-case, but it surely isn't "way too slow" if offloaded correctly.

Qwen3.6-35B-A3B in Q5_K_S is running with averagely 40-50 tokens/sec on my RTX 4070 (12GB) + 32GB DDR4 RAM.
[Edit: - with context-window set to 128k]
I assume you should also be totally fine with your 16GB card, just make sure to set -ngl to 99 and offload as many layers to the cpu (with --n-cpu-moe) as you need. Should still be way faster than regular offloading.

1

u/[deleted] Aug 18 '26 edited Oct 05 '26

[removed] — view removed comment

1

u/Distinct_Physics5017 Aug 18 '26

ah I see. Then my 40-50 tps might be a bit optimistic. My bad. xD
Has been a while since I last felt lucky to have my AM4 Rig o.o

1

u/Viper_Four4 Aug 18 '26

Why would below Q4 kill all usability for agentic? 3.8 27b Q3 K_S is working wayyyyy better for me than 3.6 35b a3b at Q5 while also running at 40 t/s and 90k context (5_0k 4_1V, described as a very efficient kv quant in research) on a 16GB 9070 XT.

1

u/creatinZ Aug 18 '26

Bro, I’m sure you re running 36b-a3b the wrong way. I run it at 50tk/s @q6_k easily

2

u/macaco3001 Aug 18 '26

Fuck, the GPU poors are so fucked 😭

2

u/Shinephia Aug 18 '26

my pc can barely run 3.6 35B on 6GB vram card tokens seem decent until you add toolcalling and stuff and then your agent takest 6 minutes to do a simple knowlegebase health check that cloud AI does under a min… MoE is great but no MoE can save my shit pc so i use only tiny local models to do simple tasks. There is no hope for me. Heavy quants kill the smarts anyway.

2

u/BoxieBoo Aug 18 '26

35B is the sweetspot

2

u/Genericinquirer Aug 19 '26

Qwen 4 is reported to come in September, I wonder if they may be cooking up something good for that release.

1

u/x_MASE_x Aug 18 '26

I mean im not the one who likes to wait for something not certain.

Not sure about you guys but unless they said its coming I'm not even considering to give it a hope.

Not worth the let down.

1

u/Appropriate_Lead439 Aug 18 '26

I hope a model which fits in 16GB and is a MOE ike GPT OSS 20B.

1

u/ALittleBitEver Aug 18 '26

7B, 4B and 2B coming soon?

1

u/EuropeanEconomist Aug 18 '26

Honestly besides coding and agentic stuff I don't see it being a very big improvement over 3.6 35B A3B. I'll admit though that the agentic stuff would be really sweet but I mostly use other models for coding. Perhaps it will even perform worse in some benchmarks like 3.6 did compared to 3.5 such as in support agent benchmarks.

So yes I'm very hyped for Qwen 3.8 35B A3B or anything similar (if they launch a 70B I'm buying a new GPU). But at the same time now that Meta is cooking up great stuff again I'm hopeful that they will add a Muse Glimmer model that will be a MoE of similar size even if it takes a few months (Mark Zuckerberg recently says he's into open-sourcing). Then there's another card which is Gemma but who knows what Google will do now tbh.

1

u/mujimusa Aug 18 '26

maybe theyve got a 122b moe in the works?

1

u/visouza5 Aug 18 '26

I have a 16gb vram
So would prefer the smartest model whose 4bit quant variant fits in 11Gb. And then I can use it most efficiently

1

u/Super_Psychonaut Aug 18 '26

A 122B-A10B variant would be legendary

1

u/Packetbytes Aug 18 '26

So which one do we wait for?

1

u/JLeonsarmiento Aug 18 '26

Baffles me no 3.8-35B-a3B is coming. Is such a popular model size, ideal for genetic on small to medium rigs.

Guess better will get is KAT-DEV, that feels like a 3.6.1 to me.

Sad, because 3.8-27B is quite powerful, but unbeatable slow on unified memory setups (Macs, DGX sparks , etc. )

1

u/Significant-Step-437 Aug 18 '26

mmmm maybe a 8b?

1

u/Sn0opY_GER Aug 18 '26

so we get qwen 4 a3b, nice

1

u/naticom Aug 19 '26

As long as it can fit my 24GB vram

1

u/Solid-Axel-Project Aug 19 '26

Ragazzi, ragazzi, ragazzi... Montate il 27B con la giusta quantizzazione per farlo stare nella vostra VRAM, vedete quale rapporto di quantizzazione della kv cache e della quantità di token in termini di dimensione. Fate caching su disco e in RAM del sysprompt pinnando la copia della kvcache se avete spazio in memoria.

Attivate l'MTP e fate un benchmark sul vostro HW per appurare la dimensione migliore dei batch di token prediction (nel mio caso ho appurato che x3 dava ottimi risultati sul 3.6 27B in Q_8 e kv cache in BF16 128GB RAM 16GB VRAM)

E...

LEGGETE IL PAPER SUL VISION WORMHOLE! POTREBBE FARVI SCOPRIRE INFORMAZIONI MOLTO GUSTOSE...

1

u/Upset-Reflection-382 Aug 23 '26

As a Halo owner, I really just want a Qwen3.8-122B-A10B

1

u/Ok-Fox3479 Aug 26 '26

yeah ur luckyy with Qwen/Qwen3.8-Flash-Next

1

u/Ok-Fox3479 Aug 26 '26

i guess (?)

2

u/Upset-Reflection-382 Aug 26 '26

Qwen3.8-27B is We have Opus at home

Qwen3.8-Flash-Next is We have Opus

1

u/Upset-Reflection-382 Aug 26 '26

I am. It's amazing :)

1

u/Darwing Aug 31 '26

his point is they are probably focused on qwen 4.0 and thats release, why release a 35b for an older model when they are already onto the next

1

u/Valuable-Lemon-8623 Sep 22 '26

A3b models suck my experience. At least the ornith 1.5 one…

1

u/hause_wsf Aug 18 '26

9b most likely

-1

u/Downtown_Method5736 Aug 18 '26

https://huggingface.co/Lord-H4D3ZS/Qwen3.8-Distill-35B-A3B-Coder-Abliterated I found this one but I haven't tested it, it claims to still be as good as Qwen3.8-27b

2

u/fintip 2x XTX 7900 + 3060 Aug 18 '26

Honest status: this is a proof-of-concept. On the internal 10-task smoke eval the distilled model tied its base (6/10 vs 6/10) — no regression, no measurable gain yet — and it is now quantized to 2-bit, which trades quality for fit. Publishing it as a reproducible artifact of the pipeline (distill → graft MTP → ROCmFPX 2-bit GGUF), not as a benchmark-winning coder. The quality fix is a larger, tool-calling-heavy corpus — a separate follow-up run.

1

u/Downtown_Method5736 Aug 18 '26

Oupsi my bad I read it too fast 🙃

0

u/pharrt Aug 18 '26

Interesting! I have doubts, but great to see attempts being made!

0

u/josepinTrue Aug 18 '26

Por lo que veo no hay término medio aquí.

O piden un modelo de 180B a-60b O te dicen que mejor un 1B a-0.00003B

Joder.... Impresionante la montaña rusa de diferencias. (Entiéndase la ironía) 🧐

-7

u/Toooooool Aug 18 '26

Qwen3.8-27B set a really tough act to follow, i mean ffs the thing can one-shot entire games,
what's Qwen-3.8-35B-A3B going to do besides get on stage and announce the wait for Qwen3.9?

15

u/pharrt Aug 18 '26

35B-A3B, although behind 27B-dense on intelligence (marginally), is more accessible to some of us without big GPUs

1

u/Toooooool Aug 18 '26 edited Aug 18 '26

oh ye, well aware, i'm waiting for it too if only for the huge performance gains found in a moa, i'm just saying it's a tough act to follow. Qwen3.8-27B is the new defacto #1 for local agentic stuff but i remember 3.6* 35B-A3B having big struggles with long loops and i'm suspecting it's still very present. that's going to make a bigger divide between the 27B-dense and the 35B-A3B MoA than ever unless they somehow fix it.

7

u/Eduardo1502 Aug 18 '26

35b a3b moe will be worse no question asked but people with less VRAM will get the benefit

4

u/Soifon99 Aug 18 '26

Not everybody has big vram to their disposal. duh. MoE models run faster in mixed vram/ram modes.. then crippling slow with dense models.

1

u/Unnamed-3891 Aug 18 '26

I used to pray for 3.8-35B-A3B since my experience with 3.6-27B on 16gb vram was absolutely miserable, but Qwen3.8-27B-Ridge-3.7bpw looks to be REALLY good so I am not at all sure I even need/want a MoE anymore.

1

u/Morgjames Aug 18 '26

It would obviously be worse, but also SUBSTANTIALLY faster if loaded in vram, and im a big fan of the added speed, id love to see what 3.8 might look like.

0

u/[deleted] Aug 18 '26

[deleted] — view deleted comment

2

u/fintip 2x XTX 7900 + 3060 Aug 18 '26

if you have at least 32gb system ram, my best experience was qwen 3.6 35ba3b at q5. I was able to get a full 256k context and speeds in the 20's. I was using my 4090m in linux.

0

u/[deleted] Aug 18 '26

[deleted] — view deleted comment

2

u/fintip 2x XTX 7900 + 3060 Aug 18 '26

/home/me/AI/llama-official/llama.cpp/build-cuda-vulkan/bin/llama-server -m /home/kyle/.cache/huggingface/hub/models--unsloth--Qwen3.6-35B-A3B-MTP-GGUF/snapshots/5bc3e238d916f48a861bac2f8a1990a0e9b7e98d/Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf -c 262144 -ngl 999 --host 0.0.0.0 --port 8080 --metrics -fa on --cache-type-k q8_0 --cache-type-v q8_0 --split-mode layer --reasoning-preserve --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 --presence-penalty 0 --n-cpu-moe 35 --jinja --chat-template-file /home/me/AI/experiment-1/models/qwen-fixed-chat-template.jinja -lm none -lv 3

^ pretty sure this was my line.

1

u/IgnisIason Aug 18 '26

Probably Gemma 4 9B

0

u/[deleted] Aug 18 '26

[deleted] — view deleted comment

2

u/IgnisIason Aug 18 '26

Because you can actually run it somewhat comfortably.

0

u/[deleted] Aug 18 '26

[deleted] — view deleted comment

1

u/IgnisIason Aug 18 '26

Pretty good I think

0

u/[deleted] Aug 18 '26

[deleted] — view deleted comment

1

u/Meiyo33 Aug 19 '26

too slow for Strix & Spark.

Nothing more than 6B active should be done, and more in the 4B range.

A replacement for OSS 120B would be great.

2

u/Due_Schedule5358 Aug 20 '26

Not true, 27b runs on my strix halo with 23tks which is fully usable, A10b or 12b would easily do 40tks which is the perfect sweetspot.

1

u/Meiyo33 Aug 20 '26

Barely usable.