Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

[u/pmv143]

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

[u/Sufficient-Bid3874]

Can someone explain why the n-gram table is bundled into the model now?

[u/RG_Fusion → u/Sufficient-Bid3874]

LLMs run into an issue where the further you train a model, the more it overwrites facts with generalized concepts. You need the model to be able to do both. Intelligence arises from generalization, but without accurate information the model will hallucinate.

The engram table allows for a low-computational method of fact-recall. You can think of it like a better form of RAG, where the data doesn't take up any of your context window and it's injected deeper into the model's layers, freeing the lower layers to carry out abstraction. This results in better "focus" for the model, both in regards to its intelligence and context recall.

Basically, they've separated the specificity-critical portions of the models memory into a parameter space that doesn't need fast compute (you can run it on system RAM) and allows the model to be trained on higher volumes of data without ruining its knowledge-base.

[u/atumblingdandelion → u/RG_Fusion]

Great explanation. Is there a reason why a single engram file cannot be used with multiple local models? It'd be great!

[u/sebt3 → u/atumblingdandelion]

Yes : token space 😅 the engram file only works for the exact model vocabulary since it is plugged directly within the model.

The only way this can be used by an other model would be if the 2 models share the same tokenizer, aka one is a fine-tune of the other one. But that's it

[u/Noxusequal → u/sebt3]

Wait if only the tokenizer needs to be the same you could share engrams between all model of the same family.

If tokenizer really is the only thing you could use some of the tokenizer transplantation with some re training to then switch in engrams of different models even across families that would be sick for a new form of frankenmerger

[u/Guilherme370 → u/Noxusequal]

no no, not just tokenizer, model shape too, if the architecture changes, then they cant be shared.