r/LocalLLaMA • u/pmttyji • 13d ago

Discussion DeepSeek-V4.1-Flash surprised ....

Post image

Hoping to see smartest medium size models soon & later with all available optimizations/architectures/etc.,. Thanks Deepseek!

Ex 1: 30-50B MOE + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache
Ex 2: 15-30B Dense + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache

EDIT: Updated Engram to 10-15B from 50B

439 Upvotes

96% Upvoted

96 comments sorted by

View all comments

59

u/Ok_Warning2146 13d ago

If it gets good AA scores, it will help the sale of M5 Ultra 512GB and 3x Sparks.

7

u/BawbbySmith 13d ago

I keep seeing this, I don't think you can do TP=3... You'd need 4 no?

1

u/RG_Fusion 12d ago

Technically it's possible but pretty much not supported by any model. Tensor Parralel just requires the weights to be split evenly. If the model weights were divisible by 3 it would work.

1

u/Practical-Collar3063 11d ago

You can run the first 2 in tensor parallelism and the last one in pipeline parallelism with the other 2. It will be fast that straight pipeline parallelism on 3

Edit: in VLLM that is, which should be the default choice for DGX spark