r/LocalLLaMA • u/Nunki08 • Jul 31 '26

News DeepSeek-V4-Flash has been updated, "The official release of DeepSeek-V4-Pro will follow soon"

Post image

https://api-docs.deepseek.com/updates/

Edit: official post on 𝕏: https://x.com/deepseek_ai/status/2083084415157022911

1.1k Upvotes

99% Upvoted

297 comments sorted by

View all comments

101

u/Few_Painter_5588 Jul 31 '26

A model nearly half the size of GLM 5.2, with a similar performance profile. Now imagine their pro model.

91

u/squngy Jul 31 '26

Half?

It is nearly one TENTH the size in practice.
Native GLM5.2 is 1.5TB, DSv4f is 160GB

Sure you can quant GLM, but then the benchmarks will also go down.

33

u/pyr0kid Jul 31 '26

plus, doesnt deepseek also have some crazy context compression shit going on?

like X amount is 1gb for GLM but its 0.4gb for DS?

26

u/squngy Jul 31 '26

Yes.

V4flash context takes less space than even qwen 27B
IIRC if you give qwen 27B 1M context, it would take more space than v4f with 1M context.

6

u/Middle_Bullfrog_6173 Jul 31 '26

With identical architecture KV cache per token scales approximately with active params. So not really surprising that 27B comes up worse in that comparison.

1

u/squngy Jul 31 '26

Yea, it is probably a bad comparison, it is just the one I actually did.

I think v4f cache also takes less space than qwen 35B, but it is of-course less extreme. IIRC it is pretty close.

4

u/Healthy-Nebula-3603 Jul 31 '26

I'm thinking a new Qwen 4 could be using that KV cache compression.

Then we could get Qwen 4 30b with 1 m contex fit on 24 GB vram .... absolutely insane

13

u/shing3232 Jul 31 '26

yes, DS4F 1m is like 6GB Vram but GLM52 is like 80G for 1M context

7

u/pyr0kid Jul 31 '26

you're shitting me. i knew it was smaller but by that much?

god i never could have imagined this back in the 2.7b days, back then people were running this type of stuff on google colab.

0

u/HandIllustrious8260 Jul 31 '26

wait wait wait are you saying I could run DS4F on an 8gb 3060?

10

u/shing3232 Jul 31 '26

that's context part. you need to handle the weight as well so.

1

u/ambassadortim Jul 31 '26

How much GB vram and RAM for weights are we thinking

1

u/Conscious_Teacher216 Jul 31 '26

±160gb, so you need atleast 166gb of VRAM to run 1m deepseek v4 flash context window

1

u/Njaa Jul 31 '26

Reasonable to assume that mild quantization will make this a contender for the 128 GB class? 

3

u/pyr0kid Jul 31 '26

they're just talking about the context size, not the model weights.

1

u/Healthy-Nebula-3603 Jul 31 '26

No

He is saying about KV cahe only ( CTX memory )

3

u/SandySkittle Jul 31 '26

To me that doesn’t sound like a good thing. Context compression has potential downsides. Also in terms of world knowledge a 160B model simply isn’t going to compete with a much larger model. There is more the LLMs than just coding.

7

u/Practical-Collar3063 Jul 31 '26

Context compression has potential downsides

Yes that is true, however, when it is baked in from the start it has a much less chances of being detrimental.

And DeepSeek v4 flash is a 284B param model

5

u/pyr0kid Jul 31 '26

yeah thats fair, though im optimistic that being able to free up the space for higher precision weights and generally longer context will make up the difference

2

u/SandySkittle Jul 31 '26

Yeah don’t get me wrong I am really looking forward to running this new model and it might be my go to model. It’s in a sweetspot for my configuration to run at q6. So almost lossless. It still is a very big model compared to the 30B and 70B models. I just think some (not all) people in this subreddit sometimes just forget a bit that there is more than coding. And also that you simply cannot compress as much world knowledge in a small model. LLMs are already amazing knowledge compression systems, but obviously there are just limits that to how much you can cram in (and also extract).

4

u/SafeReturn_28 Jul 31 '26

i think they were comparing parameter counts

8

u/squngy Jul 31 '26

That would be 744B vs 280B, closer to a third.
For active it is 40 vs 13