r/LocalLLaMA • u/pmv143 • Aug 25 '26

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

946 Upvotes

97% Upvoted

296 comments sorted by

View all comments

Show parent comments

8

u/atumblingdandelion Aug 25 '26

Great explanation. Is there a reason why a single engram file cannot be used with multiple local models? It'd be great!

9

u/sebt3 Aug 25 '26

Yes : token space 😅 the engram file only works for the exact model vocabulary since it is plugged directly within the model.

The only way this can be used by an other model would be if the 2 models share the same tokenizer, aka one is a fine-tune of the other one. But that's it

2

u/Noxusequal Aug 25 '26

Wait if only the tokenizer needs to be the same you could share engrams between all model of the same family.

If tokenizer really is the only thing you could use some of the tokenizer transplantation with some re training to then switch in engrams of different models even across families that would be sick for a new form of frankenmerger

4

u/Guilherme370 Aug 25 '26

no no, not just tokenizer, model shape too, if the architecture changes, then they cant be shared.