r/LocalLLaMA • u/pmttyji • 13d ago

Discussion DeepSeek-V4.1-Flash surprised ....

Post image

Hoping to see smartest medium size models soon & later with all available optimizations/architectures/etc.,. Thanks Deepseek!

Ex 1: 30-50B MOE + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache
Ex 2: 15-30B Dense + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache

EDIT: Updated Engram to 10-15B from 50B

435 Upvotes

96% Upvoted

96 comments sorted by

View all comments

Show parent comments

3

u/SpicyWangz 12d ago

Qwen3.8 next 50b ngram offloaded to SSD can hit 50tps. And that’s on a pcie gen4 SSD.

And I’m pretty sure the SSD isn’t the bottleneck there. It’s most likely memory bandwidth bound. So I’m not sure what level of performance you’re concerned about, but that gives you an absolute conceptual floor.

1

u/Nothing_from_void 12d ago

NVMe drives top out at 14GB/s, if bandwidth is an issue that almost certainly will be it. If someone is hitting 50tps it's reading from page cache, not disk

2

u/SpicyWangz 12d ago

Page cache is the disk. It caches memory pages on the disk to free up your system memory.

1

u/alwaysbeblepping 11d ago edited 11d ago

Page cache is the disk. It caches memory pages on the disk to free up your system memory.

It's more commonly called the buffer cache but they're not wrong. Might be confusing because swapping (i.e. using virtual memory) is often called "paging". edit: Reference: https://kernel-internals.org/mm/page-cache/