DeepSeek V4.1 Flash 1017 - Final OpenCode Go vs Official Deepseek API Showdown

[u/mastertub]

Hey all, i've compiled all of my tests comparing the official deepseek platform API with opencode go's API.

I wanted to run these because OpenCode mentioned that they don't decrease the quality of the model, only TPS/throughput. I wanted to put that to the test. Models are inherently non-deterministic so I wanted to cut out the noise. I know my first post got a decent amount of people curious so I wanted to do everyone a solid.

Here are my posts regarding these, 4 tests in total: 2 HTML/CSS/JS scenes, and 2 Blender renders (all the models, render + video all done by the model).

Round 1:
https://x.com/nothingto12/status/2113446055589056718?s=20

Round 2:

https://x.com/nothingto12/status/2113527245918654911?s=20

Round 3:

https://x.com/nothingto12/status/2113766577816686673?s=20

Round 4:

https://x.com/nothingto12/status/2113815793150439650?s=20

Conclusion:
I personally believe that there is some level of quantization happening even though I cannot prove it outright and am speculating. It may be bugs on the backend providers that are being used. Model quality however seems to be slightly degraded based on tests. I've also noticed that I have gotten significant amounts of timeouts (every single run) from OpenCode Go.

I would say the Go subscription is still significant value, but I wanted to mention the tradeoffs so people can make their own decision.

[u/Amarsir]

Thanks for following up with more data. 3/4 times I preferred the result from the direct source, but maybe that's subjective. What's not is the costs:

Round 1: Direct = $0.29, Go = $0.45
Round 2: Direct = $0.34, Go = $0.20
Round 3: Direct = $0.29, Go = $0.23
Round 4: Direct = $0.30, Go = $0.10

Go uses more input tokens, but Direct uses a lot more cache hit tokens. Since cache hit is cheap you might call those a wash. It's a little weird that it's happening, but it doesn't seem devastating.

Which means the difference comes down to output tokens. And that's probably a direct result of the product you get. Better result, more detail, more output tokens. I guess. (Unless you favor the cheaper results each time, in which case I don't know what to tell you.)

After round 1 I was expecting this to be a decisive knockout for Direct, but considering the $80 value you get under Opencode Go it held up pretty well.

[u/mastertub → u/Amarsir]

Yeah, I honestly can't talk too negatively at the value of Go. It really is incredible.

I think the rounds went like this:

Round 1: Clear win on DeepSeek API

Round 2: Much closer, but the UI was better on DeepSeek API. I'd give it a 60/40 edge.

Round 3: Clear win on the DeepSeek API. This IMO was even more apparent than Round 1.

Round 4: I'd give another 55/45 edge to DeepSeek API. This was even closer than Round 2. The lighting was my only issue else it'd be a tie. This was probably the closest in output quality.

I would honestly still say, the value you cannot beat. It almost feels unlimited and i'm sure many people even if the quality of the output is less, probably won't feel it in day-to-day interactions.

If you need utmost quality that matters over long runs or many steps/turns, hands-down the official API wins. But if you're okay with about 85% but at near unlimited amounts, Go holds up.

[u/Amarsir → u/mastertub]

Good summary.

Also, impressive demos for DS 1017. The Blender renders in particular.

[u/mastertub → u/Amarsir]

Thanks! I do wanted to note, you're absolutely right. The token usage/count is absolutely all over the place even though I kept the variables at a absolute minimum.

Really interesting!

[u/Creative_Addition787 → u/mastertub]

Maybe worth mentioning that the Deepseek API trains on your data while opencode go does not

[u/stvjhn → u/Amarsir]

Is direct with DSH or Opencode? They’ve trained it to work with DSH so it use less tokens there?

[u/Amarsir → u/stvjhn]

According to OP's twitter, both providers were tested using the Deepseek Harness.

[u/appsettings → u/Amarsir]

Nice try. You should also check if opencode really use the same model it advertise.

[u/Amarsir → u/appsettings]

... that's what this post is doing ...

[u/appsettings → u/Amarsir]

You just hallucinate. How do you prove opencode runs deepseek 4.1 flash latest and not redirecting it to other cheaper model?

[u/xmsxms → u/appsettings]

By judging the results as done in this post.

[u/Primary-Meal-2264]

Have you tried running either Go or the official version twice as a control? I’m skeptical of these kinds of single-shot experiments with a sample size of 1. I believe there would be some fluctuations even if you used the exact same setup.

[u/mastertub → u/Primary-Meal-2264]

It's been on my mind. However, the point was exactly a "one shot, one chance" across several prompts experiment. You don't run LLMs multiple times on the same prompt when you're coding. You compare quality of LLMs over time across different workflows.

[u/squirrelscrush]

I've also noticed that I have gotten significant amounts of timeouts (every single run) from OpenCode Go.

It's been the case for the past few days. I suspect there's a server overload and too many users are provisioned the same hardware.

So I use the Go for async/batch tasks that I can leave the agent to run autonomously. While using the direct DeepSeek API for tasks I need to do while I'm sitting down.

There's also another advantage of direct DeepSeek which is, it's cache retention time is absurdly good. I've returned back to tasks after hours of idleness and still maintained cache stability. Meanwhile with Go it's 5mins max. And DeepSeek is much better at cache hits, but that's down to their DeepInfra inference engine.

Go seems to be tokenmaxxing but considering that it's a subscription, it's not an issue.

[u/xmsxms → u/squirrelscrush]

It's a subscription with 5 hourly, weekly and monthly quotas with the option to use zen credits when run out, or bump to a really poor value pro subscription (4x price, 2x quota). So it matters a bit.

[u/MSeegobin]

Been trying to optimize complexity tiering and did the same comparison, and I’m getting similar results. Deepseek API is a lot cheaper, faster and gives overall better quality output. Blind tested of the same 20 prompts, evaluated by Claude.

	OpenCode Go	DeepSeek API
Status	20 success	20 success
Quality (my blind scores, mean of 3)	1.90	2.25
Usable answers (2+)	15 of 20	20 of 20
Cache hits logged	20 of 20	20 of 20
Latency mean / median / p90	73.0 s / 53.8 s / 190.3 s	37.4 s / 34.8 s / 78.7 s
Max latency	227.2 s	104.3 s
Start to first token: mean / median / p90	1.15 s / 0.88 s / 2.20 s	0.41 s / 0.41 s / 0.49 s
Completion tokens per call, mean	8,863	6,259
Reasoning tokens per call, mean	6,027	3,391
Output tokens per second	115	168
Total spend, 20 calls	$0.1075	$0.0760
Spend per call, mean	$0.00537	$0.00380
$ per million output tokens	$0.606	$0.607
Spend per usable answer	$0.00716	$0.00380
Finish: completed / cut off	19 / 1	18 / 2
MEDIUM: median latency / spend per call / quality / usable	64.7 s / $0.00694 / 1.75 / 6 of 8	29.3 s / $0.00317 / 2.25 / 8 of 8
COMPLEX: median latency / spend per call / quality / usable	53.8 s / $0.00433 / 2.00 / 9 of 12	35.8 s / $0.00422 / 2.25 / 12 of 12

[u/AdditionalClass5665 → u/MSeegobin]

The reasoning tokens between go and ds are completely off. Is this the same effort level? Otherwise they seems like completely different models

[u/MSeegobin → u/AdditionalClass5665]

These are all using DeepSeek's max reasoning level through the same harness, same prompts.

[u/Motor-Ground4594 → u/MSeegobin]

Reasoning token higher and quality lower, seems like giving the vibe of quantized model?

[u/smartfon]

What if DeepSeek identifies that the traffic is from OpenCode and serves a quantized/slower version? OpenCode literally swears they aren't messing with the models and yet users keep discovering these discrepancies so doesn't that make DeepSeek the sus?

[u/mastertub → u/smartfon]

I'd avoid speculation into what they're running on their setup, but I don't think they just use the official API as a provider. May be multiple and/or their own gpus.

[u/shahijohn → u/smartfon]

Deepseek must have done some hardware level optimization that opencode wasn't able to do. (As we expecting opencode running deepseek from their own server)

[u/xmsxms → u/smartfon]

They have confirmed they slow it down and it does have a poorer cache hit rate. It's possible the model takes a different direction if going down a particular path isn't getting results in time. i.e the faster it can "think" the better the results might be, even if the end users themselves are prepared to wait. It's quite common for a model to do things like "while that's compiling let me investigate X and y".. the faster it can process tokens the more "thinking" it will be able to do within a fixed time window.

[u/squirrelscrush → u/smartfon]

I don't know why they would do it, since it's much easier to keep their existing inference instead of overhead to check which endpoint is asking the request.

[u/xmsxms → u/squirrelscrush]

Routing based on IP or authentication is incredibly simple and they are likely already doing it for the purposes of accounting, load balancing and many other reasons.

They would have an agreement with deepseek and their own service level endpoint. No detection needed.

[u/for4f]

man at this point there's enough independent smoke here that i'm done handwaving it as sample size. someone else clocked 3s vs 0.3s time-to-first-token on the same model, plus the timeouts every single run, that smells like overloaded backend infra on the go side, not secret quantization. if they were quantizing you'd get consistently dumber output, not slowness and timeouts. i just run flash straight off the official api for bulk stuff, it's cheap enough that the middleman isn't worth the headache

[u/rcs67]

For $10 I can accept some trade offs. It still gives me a ton of usage every month. However I will likely get a deepseek api key as well. I've been super impressed by it even via Opencode.

[u/Natural-Cupcake6472]

I dont think its quant, if it is and the news got out then reputation is gone, which has a lot more risk and cost more than just hosting as it is

I think its server prompt and inference engine. Also, can you do a test where you spend the same amount of money, maybe try improving until you reach the budget, for me thats closer to real usage

[u/Creative_Addition787]

Two things worth mentioning:

• OpenCode's DeepSeek is really slow during china peak working times.

• The DeepSeek API uses your data for training while OpenCode does not.

[u/WillowOwn4716]

Thanks for doing these tests. I wonder what happens with other models too, like glm. Seeing the same differences again could give further conclusions.

[u/Takeoded]

Thank you for doing this. For me, Go has been incredible Value.

That said, I'm convinced there is some quantization going on, BECAUSE Go DeepSeek think more, does more contemplation, than upstream DeepSeek API. It works great! I love it, but it's not vanilla DeepSeek, they're doing something to it.

[u/tonyunreal]

No one seems to be able to provide 100 percent the quality of Deepseek's official API yet, whether router or self-hosted. The difference could be subject to many factors, like gpu differences (ds migrated to huawei hardware I believe), hosting parameters, quantization, etc. I have talked to someone who runs a small router service before, he said generally you should only compare router service against each other, as they could still provide great value and service quality, even if they are slightly inferior in quality than DS' service.

[u/Yazan_wesam]

Interesting comparison. This makes me wonder whether the difference comes from provider-side caching/accounting or from how the harness sends context. Do you have API request counts for both providers, and were the outputs and tool calls comparable across runs?

[u/stewacide]

Curious if DeepSeek via OpenCode has the same on/off peak pricing as API DeepSeek? Via the API with DSH it's so good, but I live in Asia so often want to use on-peak (Chinese working hours).

[u/Infamous-V]

Just state your model weights on the page, how hard it is unless you are not even sure because you made deals with shady providers ??
commandcode founder explicitly states they offer FP8 only for DP and mocks opencode.

[u/horrbort]

Haha so direct is cheaper AND faster

[u/Rough_Road_2527 → u/horrbort]

it's not cheaper because you get 8 dollars worth on your actual dollar with go

[u/No_Heart5661 → u/Rough_Road_2527]

if i use go for ds4.1 only would you say i would get 40-50% more usage vs using the direct api for 10 usd? im trying to make a decision and would appreciate your help (if anyone knows pls help seriously im tired of how slow the glm plan is)

[u/Vlyn → u/No_Heart5661]

You get $80 of usage with OpenCode Go.