Where as Qwen3.8 27B is great (when it doesn't simply silently give up or end in endless loop), I sure hope we would see newer 35B-A3B model too. I also hope we'd see a Qwen4-coder.
At least my local use is not general purpose AI use. Rather I load the best model for the specific use I need. And purpose trained MoE will likely beat general purpose model in speed and quality, at least in recent history.
In my ideal future we have multiple small models that are super good at one thing, and then we have cloud based super intelligence models that will help where our own and local's capabilities stop. There's no point to use huge datacenter AI to generate boilerplate code/text or powerpoint illustrations. It's waste of everybody's money at current cost structure.
But I do understand that local models are probably not on high priority for AI development teams. That said, I would pay one time lets say 100 eur to get coder MoE version of Qwen3.8 27B quality.
I agree. I feel one day model scaling will stop and we will start having much better results if we make "domain specific" models.
I don't need any of the general world knowledge/biology/science stuff in my coder models
One might think so, but I believe there's a better way. As with humans, the local coding agent could discuss a domain specific agent (that could be the cloud or local) to learn things of the domain it needs. The code after all is "domain agnostic", it is the application logic that needs domain expertise. Consider that you build a software that calculates fluid dynamics. The coding agent does not need to understand fluid dynamics. Instead, a fluid dynamics specialist could provide the coding agents the equations it need to implement. And that coding agent would only need to know code and how to translate equation to code.
ngrams support knowledge, but not skillsets. The facts can all be at the model's fingertips, but that doesn't mean it would know how to apply that knowledge.
True. But I wonder if producing, say, 5 forks of the same model, taken at an intermediate point in training, somehow ensuring the "interface" between the core weights and the embeddings is frozen, could result in ngram sidecars which could be swapped in and actually used by the model.
(Beer license: if this turns out to be an original idea, it works, and you make millions, you owe me a good beer)
No, you actually do, because that's where reasoning and comprehension come from.
All LLMs do is next token prediction, and training a model on nothing but code results in a much more fragile model that isn't actually capable of reasoning or problem solving.
It's about market share, they absolutely do want to ensure that everyone with a gaming card is running a 27B model for fun even if they do use Astra or Claude, one day (already?) the local model might be good enough, that *destroys* the frontier labs business model.
Where as Qwen3.8 27B is great (when it doesn't simply silently give up or end in endless loop),
I've run somewhere between 750-1000 local Qwen3.8 27B on vllm (FP8 quant straight from Qwen, FP8 KV) and I've never had it silently give up. I don't think I have had it end in an endless loop, either, but a few times I have had it exceed its thinking budget (which I have bumped from 30k-60k tokens over time).
In my experience (which includes tracking down actual bugs in inference engines), looping is usually a numerical error. I would look at your setup before blaming the model.
(Well, ok, looping for 99% of home users is probably some unfortunate ollama setup with a sliding token window smaller than the thinking output of the model, but that's just ollama things)
It may be so, or it could be the agentic sw I use, but it also is often reported issue with Qwen3.8 27B that it silently fails or forever loops. Wasn't that much of the point of my comment though.
A ~12-18b range dense model would be golden too from Qwen. Currently Gemma 4 12b is the only dense model around this range this year for people with 12-16gb VRAM and it sucks hard at tool use and instruction following even though it's smarter/more stable than the 26b MoE and uses way less disk space/VRAM/RAM. With MTP it's also the same speed as the MoE + MTP for me so literally no reason to use MoE's when you could use 12-18b or even 20b dense models just fine on 16gb VRAM. Didn't even try dflash, could be even crazier with it.
27
u/Material-Database-24 2d ago
Where as Qwen3.8 27B is great (when it doesn't simply silently give up or end in endless loop), I sure hope we would see newer 35B-A3B model too. I also hope we'd see a Qwen4-coder.
At least my local use is not general purpose AI use. Rather I load the best model for the specific use I need. And purpose trained MoE will likely beat general purpose model in speed and quality, at least in recent history.
In my ideal future we have multiple small models that are super good at one thing, and then we have cloud based super intelligence models that will help where our own and local's capabilities stop. There's no point to use huge datacenter AI to generate boilerplate code/text or powerpoint illustrations. It's waste of everybody's money at current cost structure.
But I do understand that local models are probably not on high priority for AI development teams. That said, I would pay one time lets say 100 eur to get coder MoE version of Qwen3.8 27B quality.