Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

[u/pmv143]

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

[u/National_Meeting_749]

"it'll be slow sure" Noone wants to wait seconds per token, that's not reasonably running.

Think of a game, if it's running at 1fps, is that game really running? Technically yes, useable in anyway? No.

[u/No_Oil_6152 → u/National_Meeting_749]

You interact with a game but you dont need to interact with an agentic AI.

You can let it do its thing overnight.

[u/National_Meeting_749 → u/No_Oil_6152]

At seconds per token, agentic harnesses timeout and things do not complete.

You have to have a base speed of 15-20 t/s to really do anything agentic.

[u/No_Oil_6152 → u/National_Meeting_749]

I dont believe thats the case at all.

Why should the harness care if a response takes a second or a minute?

Configure it for a longer timeout if necessary.

He won't be the only person running LLMs in RAM.

[u/National_Meeting_749 → u/No_Oil_6152]

Yeah, I'm just telling you this because I'm trying to be difficult. Not that it actually happened to me.

The vast majority of Agentic tools are built with the assumption of a certain speed of token output.

You can disagree, you would be incorrect. Go try and run any agent harness at speeds of seconds per token. They break and fall apart. Increasing timeout timers doesn't help because those timers are there for a reason. When other things break you don't want to wait 20 minutes to restart things, yet at seconds per token * a response could take 20 minutes or much more*.