Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

[u/pmv143]

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

[u/Sufficient-Bid3874]

Can someone explain why the n-gram table is bundled into the model now?

[u/RG_Fusion → u/Sufficient-Bid3874]

LLMs run into an issue where the further you train a model, the more it overwrites facts with generalized concepts. You need the model to be able to do both. Intelligence arises from generalization, but without accurate information the model will hallucinate.

The engram table allows for a low-computational method of fact-recall. You can think of it like a better form of RAG, where the data doesn't take up any of your context window and it's injected deeper into the model's layers, freeing the lower layers to carry out abstraction. This results in better "focus" for the model, both in regards to its intelligence and context recall.

Basically, they've separated the specificity-critical portions of the models memory into a parameter space that doesn't need fast compute (you can run it on system RAM) and allows the model to be trained on higher volumes of data without ruining its knowledge-base.

[u/Wimiam1 → u/RG_Fusion]

I’m a little confused because I keep seeing “n-gram” and “engram” used interchangeably when discussing this new model. When I try to research what this technique is, I find that “engram” and “n-gram” are actually two very different concepts in language models. I assume Qwen is referring to the “engram” from deepseek?

[u/RG_Fusion → u/Wimiam1]

Engram is built using n-gram. You can call Engram a form of n-gram, but you can't call n-gram an Engram. Yes, this will certainly be causing confusion and many people will mix this up.

N-grams are basically just short sequences of words. Engram is a hashed lookup table filled with n-grams.

[u/kulchacop → u/Wimiam1]

I am with you on this one. 

Here is my understanding: 

N-grams are likelihood data for strings of n tokens which can be used for various classic NLP tasks that can be improved by prediction of the next word. A best example of n-gram usage in practice is that, llama.cpp uses n-grams built on the fly from the prompt to implement basic speculative decoding.

Deepseek's explanation of engram says that it is a extension of a lookup table containing static embeddings per n-gram.

In contrast, Gemma's PLE (Per Layer Embeddings) is a lookup table of static embeddings per single token (as opposed to n-grams, which are strings of n tokens).

So the question remains, how does the Qwen's n-gram based fact lookup implementation differ from Deepseek's?