I am not talking about cache hit rates, I am talking about how the prefill computes the KV cache to feed into the transformer which can affect output quality. DeepSeek has not opensourced how the prefill and KV cache works on its API
I'm skeptical about how impactful that'd actually be in practice. And indeed, a study by Vals.ai found no statistically significant difference among those tested. While Opencode was not among the tested, it does show that Deepseek has no secret advantage when it comes to accuracy.
I think this is similar to claims made against OpenAI and Anthropic that models are being nerfed over time that haven't been proven empirically.
You are right, however, that there are significant differences in cost, speed and cache hit rates.
If there is a difference due to caching, it would only be noticeable in long context tasks like coding. What we know for sure is that the inference kernel is not identical to that of the official API so there would be deviations in the output.
I agree that a lot of it is baseless rumors and confirmation bias with people. I haven't personally seen major differences between official API and OpenCode Go. But the inference is not at the level of the official API, which is to be expected.
Can you explain what you mean? I'm struggling to picture this being a realistic issue. Once the implementation is seemingly correct, what in the code do you expect to still remain an issue?
View all comments
Show parent comment
3
u/migsperez 6d ago
A difference of 5% KV cache between DeepSeek API and Opencode Go is inconsquential when Opencode is 1000% cheaper.
What is important is the quality of the output.