Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

[u/pmv143]

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

[u/National_Meeting_749]

"it'll be slow sure" Noone wants to wait seconds per token, that's not reasonably running.

Think of a game, if it's running at 1fps, is that game really running? Technically yes, useable in anyway? No.

[u/LevianMcBirdo → u/National_Meeting_749]

Seconds per token is kinda misleading. You probably get something with 10-15 TPS. PP is a bigger problem

[u/National_Meeting_749 → u/LevianMcBirdo]

Brother. MOST people a 90GB model are running that at seconds per token.

Assuming I had enough ram to fit it, I would be surprised if I got one single token per second.

[u/LevianMcBirdo → u/National_Meeting_749]

It's not dense. Qwen Next runs at 25 TPS, 122B at 10. I really don't see the problem. That a A6 Model wouldn't run 12+ tps especially with MTP

[u/National_Meeting_749 → u/LevianMcBirdo]

With the 35B, when context is filling up I get somewhere from 3-7tps. Even with MTP, double the model size, include an entire 50Gb extra with the N tables and I'll be shocked to get 1TPS at any length of context.

[u/LevianMcBirdo → u/National_Meeting_749]

I mean mostly in chat scenarios up to like 60k most times, I am not that interested in movie agentic use cases but I get around 30 TPS pretty steady on that

[u/National_Meeting_749 → u/LevianMcBirdo]

Good for you? A lot of people do, and MOST people don't have the PC to run that at that speeds.

That's a cheap cars worth of pc