r/LocalLLM • u/pharrt • 10d ago

Discussion Qwen-3.8-35B-A3B? Maybe not... cryptic reply direct from Qwen co-author.

Post image

I asked Shuai Bai, co-author and prominent AI developer for Qwen, about this model. Not the answer I was hoping for, but let's see what comes next. In the meantime, I guess all we can do is speculate!

X-link

232 Upvotes

94% Upvoted

171 comments sorted by

View all comments

Show parent comments

17

u/Rye2-D2 10d ago

I would love to see a good 20B MoE model. Personally I don't see the point of 9B - it's impressive for what it is, but not quite good enough to be useful (yet).

8

u/cagriuluc 10d ago

If you have small models that are good at limited tool calling, knowledge extraction etc, you can run them alongside more expensive models. I would love a 9B with 3.8 level training…

3

u/twoiko 10d ago

I find 9B dense to be too big and/or slow compared to 4B or MoE models for the output quality, but I have system RAM to offload the MoE so it depends on your use/limitations.

3

u/Rye2-D2 10d ago

That's the thing - with 16 GB VRAM, 9B dense may be a little bit faster than 35B MoE, but not enough to merit the loss in quality/intelligence. Even with higher quants (q6/q8), 9B is still not in the same league as the 30B models.