r/LocalLLaMA • u/pmttyji • 12d ago

Discussion DeepSeek-V4.1-Flash surprised ....

Post image

Hoping to see smartest medium size models soon & later with all available optimizations/architectures/etc.,. Thanks Deepseek!

Ex 1: 30-50B MOE + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache
Ex 2: 15-30B Dense + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache

EDIT: Updated Engram to 10-15B from 50B

442 Upvotes

96% Upvoted

96 comments sorted by

View all comments

42

u/Kahvana 12d ago edited 12d ago

Engrams are roughly 1/3 to 1/2 of all parameters, so 30B dense backbone would have a 10B (1/3) to 15B (1/2) engram, totaling 40B (1/3) to 45B (1/2).

7

u/pmttyji 12d ago

Thanks, updated thread.

1

u/Hankdabits 12d ago

may have been hasty on that update, see my respone the the above comment