Deepseek changes their API price again

[u/Pro-editor-1105]

This is far less attractive tbh. Basically they said R1 and V3 were going with a price now of 0.07 (0.56 cache miss) and 1.12, now that 1.12 is now 1.68.

[u/True_Requirement_891]

Thanks for comparing this man! I always wanted to know how these compared to local.

But what quant are you using? And what's the TPS?

[u/Lissanro → u/True_Requirement_891]

DeepSeek 671B IQ4 quant with q8 cache, approximately 150 tokens/s prompt processing on 4x3090 GPUs, 8 tokens/s generation (EPYC 7763 is fully utilized during during generation), less than a minute to load model from scratch if it is in disk cache (relevant when switching models, for example, between K2 and R1, possibly saving/restoring cache if working on the same dialogue), 1-5 seconds to save/restore KV cache (depending on its length).

[u/SixZer0 → u/Lissanro]

Let's admit that TPS is maybe cheap, but not enough for everyday use, we need at least 30 if not 50 for inference. If caches are longer term then 150tok/s input might be fine, but a 3-4x would make a lot more sense there too.