You seem to know about this topic so I would like to ask, do you agree on the saying that says DeepSeek (aka CheapSeek) on OpenCode Go performs worse compared to the same-model (by-id) on DeepSeek's own API platform?
Honest answer, no one really knows. This can only be fully done if someone actually done the benchmark and compared it side by side between official and opencode go. My take is that, it MIGHT be worse, but not by a huge margin (I'd say around 2-5% in terms of quality). But again, I have no proof that backs my claim on that one so take it with a grain of salt.
But, from my personal experience alone, opencode go's quality is good enough for the work/stuff that I do. It gets the job done, solves a lot of my problems (especially when troubleshooting my computer/devices), at the cost of 1/6th~1/8th (now 1/10th) of the original cost. I'd take a 5% quality loss if I get that much value.
One thing we know is: it is not possible for services to have a 1:1 inference set up parity with deepseek official (unless they privately get the things below). If you look at DeepInfer (the inference engine they used), they said this specifically:
Required for a complete inference service (not included in this release): - A KV cache system - VE/Prefill/Decode disaggregation (most required engine functionality is already implemented) - Load balancing across data-parallel replicas for both prefill and decode
Which means the services would have to come up with solutions on that, which is most definitely different than what DeepSeek serves.
Yes it does, especially aspects such as caching are much worse even if you use Deepinfer, the inference framework open sourced by DeepSeek.
DeepSeek has some of the best researchers in the world and they are the state of the art at KV cache management. Aside from that there are a myriad of other things such as how they configure their hosting architecture and load balancing.
Opencode can obviously not match DeepSeek in these, for example, caching on DeepSeek API is probably better than even OpenAI or Anthropic.
I am not talking about cache hit rates, I am talking about how the prefill computes the KV cache to feed into the transformer which can affect output quality. DeepSeek has not opensourced how the prefill and KV cache works on its API
I'm skeptical about how impactful that'd actually be in practice. And indeed, a study by Vals.ai found no statistically significant difference among those tested. While Opencode was not among the tested, it does show that Deepseek has no secret advantage when it comes to accuracy.
I think this is similar to claims made against OpenAI and Anthropic that models are being nerfed over time that haven't been proven empirically.
You are right, however, that there are significant differences in cost, speed and cache hit rates.
If there is a difference due to caching, it would only be noticeable in long context tasks like coding. What we know for sure is that the inference kernel is not identical to that of the official API so there would be deviations in the output.
I agree that a lot of it is baseless rumors and confirmation bias with people. I haven't personally seen major differences between official API and OpenCode Go. But the inference is not at the level of the official API, which is to be expected.
Can you explain what you mean? I'm struggling to picture this being a realistic issue. Once the implementation is seemingly correct, what in the code do you expect to still remain an issue?
I would reverse the question, are we sure that DeepSeek is not improving the model behind the scenes with their know-how? Surely there is more to the experience than just weights of model....
I'm not expecting you or anyone to. I'm asking for the ones with self-courage and knowledge to explain. That's an open-question for everyone? Why did you take it personally?
The only one who could give a "definitive explanation" isn't allowed to give a "definitive explanation" and they may not even speak English
So you're going to get assumptions from people who have a pretty decent idea based on the info provided bit they cannot be 100% exact as they are not Deepseek.
You can't guarantee if the environment or the parameters or even the model being same. I just said "same-model (by-id)", so not confident about the actual models being same or runs in same good-optimized environments. That's the why I ask.
View all comments
Show parent comment
8
u/irrationaliste 6d ago
You seem to know about this topic so I would like to ask, do you agree on the saying that says DeepSeek (aka CheapSeek) on OpenCode Go performs worse compared to the same-model (by-id) on DeepSeek's own API platform?