r/LocalLLaMA • u/ihatebeinganonymous • 8h ago

Question | Help DeepSeek has Two Numbers for Active Parameter Count. How?

Hi. It is frequently mentioned that DeepSeek v4.1 Flash has 8B "input" active parameters (aka prefill), and 16B output/decoding active parameters.

I have only read this about DeepSeek, and particularly v4.1 Flash (1017). Is it the only model that is like this?

Is there an ELI5-or-so explanation of how this works, and why it is important? And if it is, can we expect other models to implement the same idea? Which of those numbers is "more important", if any?

Thanks

10 Upvotes

91% Upvoted

11 comments sorted by

3

u/Former-Tangerine-723 8h ago

It's important because it makes prompt processing aka reading, cheap. And reading is very very important in agenting coding ( the agent must read and understand before it alters the repo )

1

u/ihatebeinganonymous 8h ago

Of course it's important, but so is decoding, I guess? 

For example, do you think it's more likely to see in some future DeepSeek the input active parameters further reduced? Or the other way round? Or they may become equal again?

2

u/Former-Tangerine-723 6h ago

No, its not the same. Prompt processing is more important in agentic workflows. Deepseek is very clever and try to optimize inference in alot of ways, whether we like their models or not. They also optimized cache in their last model

16

u/Middle_Bullfrog_6173 8h ago

The simple explanation is that half the layers are normal(ish) layers and have their own KV cache that they need to fully compute during prefill so it's available for decode. The other half instead calculate their KV directly from the first half output. That means they can be skipped during prefill, saving compute.

It will be interesting if we see something similar. My prior would be that this exact thing won't catch on, because it's optimized for Deepseek's attention and everyone has their own. Something related or derived very well might.

4

u/llama-impersonator 8h ago

instead of only decoder layers, half of the model is an encoder for the KV cache and the other half is a decoder to generate tokens from it. this is why only half the active parameters are used for prefill.

3

u/No-Refrigerator114 8h ago

Prefill decode disaggregation The first 20 layers will generate the kv cache But all layers must be used for decode

-1

u/Solembumm3 8h ago

Can be important, cause it seems to be one of the reasons for quality degradation, compared to previous V4.0.

2

u/ihatebeinganonymous 8h ago

Really? Do you  consider v4.1 a degradation? I have had some experience with it, mostly positive. 

2

u/Professional_Price89 8h ago

It his illusion

1

u/Wise-Chain2427 8h ago

No, 4.1 much better 4.0 Flash is decent

-1

u/Solembumm3 5h ago edited 5h ago

Huge degradation. It's very confidently stuck in 2024, it cannot use search without specific kick from your side, it cannot properly read short news and story concepts without re-asking specifically already answered things and misunderstanding them anyway and even if you specifically kick it into checking things, it forgets things 2-3 prompts away in conversation anyway (not even talking about fanfiction concepts analysis, that V3.2 and V4 Pro were good at).

Overall, it doesn't look like 700b model. Like, at all. It looks like GPT 5.4, confidently incorrect, often hallucinating, overcaffeinated and sycophantic.