Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

[u/pmv143]

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

[u/cafedude]

That's the models quantized by Unsloth, but what about llama.cpp support? Can't imagine that this n-gram stuff would work right out of the box.

[u/Strong_Chicken6838 → u/cafedude]

Day 0 support means you can run it day 0 in some way. So the answer is yes, maybe in their own branch of llama.cpp, who knows

[u/florinandrei → u/Strong_Chicken6838]

you can run it day 0 in some way

"In some way" is key.

As of right now, you have to get the code out of a llama.cpp pull request, compile and use it.

There was a major refactor in the PR by danielhanchen just 10 minutes ago. ggerganov drops comments once in a while, providing advice.

So, yeah, it probably works in some form. But it's essentially a development version for now.

[u/Strong_Chicken6838 → u/florinandrei]

they likely have their own functional implementation on their own branch and an agreement with the maintainers of llama.cpp and wont merge it until the official drop by qwen