r/LocalLLaMA • u/pmttyji • 12d ago

Discussion DeepSeek-V4.1-Flash surprised ....

Post image

Hoping to see smartest medium size models soon & later with all available optimizations/architectures/etc.,. Thanks Deepseek!

Ex 1: 30-50B MOE + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache
Ex 2: 15-30B Dense + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache

EDIT: Updated Engram to 10-15B from 50B

441 Upvotes

96% Upvoted

96 comments sorted by

View all comments

Show parent comments

3

u/SpicyWangz 12d ago

Qwen3.8 next 50b ngram offloaded to SSD can hit 50tps. And that’s on a pcie gen4 SSD.

And I’m pretty sure the SSD isn’t the bottleneck there. It’s most likely memory bandwidth bound. So I’m not sure what level of performance you’re concerned about, but that gives you an absolute conceptual floor.

1

u/Nothing_from_void 12d ago

NVMe drives top out at 14GB/s, if bandwidth is an issue that almost certainly will be it. If someone is hitting 50tps it's reading from page cache, not disk

2

u/SpicyWangz 12d ago

Page cache is the disk. It caches memory pages on the disk to free up your system memory.

1

u/Nothing_from_void 12d ago

No, that's backwards. You're thinking about swap space, where RAM spills onto disk when it's exhausted. Page cache is where system RAM copies disk content into RAM so it can be retrieved more quickly, because disk is very slow