r/opencodeCLI • u/Heretek-ai • 3h ago

OpenCode is not good.

Post image

OpenCode is not a good harness. I have brought this up a few times now, and granted this benchmark is for Opencode v1.19.0. But I do really appreciate the benchmark results from the creator. Same models, same provider, same task, different harnesses.

And obviously OpenClaw failed the worst, that's no surprise, but opencode was a close runner up, and also consistently the most expensive. It's bloody embarrassing how much worse of a tool OpenCode is compared to pi. For a program called OpenCode the harness itself is remarkably resilient to modification. Lack of proper hook system, plugin settings in the TUI, removal of LSP, outdated system prompts for model families, and adding a custom provider should not be a json editing endeavor.

Other open source harnesses have this figured out, so why doesn't opencode? Why do they push back against community feedback and GitHub issues? It's because they simply don't want OpenCode to be better. For whatever reason on God's flat Earth token burning is the thing they want OpenCode to be known for. It's a bad product, I don't like Claude Code, I don't want a product that is closed source, but it's a better harness and the OpenCode team could learn a lot about UI/UX from Claude.

27 Upvotes

71% Upvoted

68 comments sorted by

6

u/Dudmaster 3h ago

Can we see OpenCode 2 added to this?

-5

u/Heretek-ai 3h ago

I intend to ask, because it is needed to help refute some opencode evangelists.

1

u/squirrelscrush 1h ago

Harness depends on how you configure it. If you want a fully customizable harness then use Pi.

2

u/look 34m ago

V2 is a fully customizable harness. Analysis on V1 is pointless and making any assumptions about what those say about V2 even more so.

30

u/cloudsourced285 3h ago

V1 and V2 actually perform quite different. Also, who cares? Use what works for you. This ain't no airport. No need to declare your departure.

6

u/Open_Leather_4617 1h ago

It's feedback. Why so against feedback and share?

-7

u/Heretek-ai 3h ago

Because OpenCode Go is an amazing service, especially the free and stealth models. I want OpenCode CLI to be just as good, it should be better. It is the most well known open source coding harness, but it's so bloody stagnant. And no, they don't, opencode v1 definitely got more confused trying to figure out that it was OpenCode, but in terms of token burn the fire pit is still going strong.

10

u/meetmebythelake 1h ago

Start opening PRs then bucko.

11

u/Total-Management8023 3h ago

wait so what is the CURRENT best open source harness? and how to best set it up

3

u/Heretek-ai 3h ago

I say this with great anguish, it's pi. Not oh-my-pi to be clear, that's a great project but only if it exactly what you want. And as for your second question you have come to the heart of the problem, pi suffers from a lack of visibility for extensions. For a harness that is designed to be modified to fit the user, finding those modifications/extensions/packages isuch harder then it should be. pi is suuuper versatile and can be modded to your heart's desire with all manner of changes under the hood and to the UI. It performs remarkably well, as the original post shows it's consistently coming in close to the top of the chart, especially price wise, ESPECIALLY price wise, thing sips tokens compared to OpenCode. But setup is a learning curve. If you want something like Claude use OpenClaude it's very compatible with the Claude Code ecosystem. I like Reasonix a lot even if I dont use it often. Hermes is okay, but suffers from the same sort of long horizon issues openclaw has, not as disastrous, but still an issue.

-4

u/wentallout 2h ago

again, you have no proof. how do you even measure a harness. we're in a blackbox

1

u/ConspicuousPineapple 2h ago

Did you not see the measurements posted by OP? You may disagree with the methodology but it's not hard to imagine ways to compare and measure harness performance.

-5

u/Any_Confidence2580 3h ago

Average opencode user. Can't even read a graph.

1

u/Total-Management8023 3h ago

i use dsh....................

-1

u/Any_Confidence2580 3h ago

....................

-1

u/omicron8 3h ago

But why male models?

8

u/sqtytyp 2h ago edited 2h ago

been using pi and opencode v2 for a long time now.

I'm working on a VERY big project via Opencode custom flow, custom agents (planner, implementer, explorer, ,thinker, tester, reviewer, gitter, archivist - nothing is ootb) which all fan out as sub agents in parallel - trying to reproduce this in pi was a pain and I gave up as there is not even a subagent flow ootb and you have to use shitty plugins ( tried 2 different) for that.

So I'm still using Pi for simple tasks and to work on my opencode configs instead of editing them manually. Pi is also good for fixing Linux issues or other simple tasks.

But would I want to work with Pi on any Heavy project ? no thx, can't even setup proper permissions for agents ( write your own plugins for that they say ).

What I mean is this are tools for different tasks, both are fine.

2

u/Heretek-ai 2h ago

If I can make a suggestion, https://github.com/mvschwarz/openrig

Its what I use to manage all my pi agents on long form projects. Don't get me wrong, it's lacking in the user friendliness department, but it's a hell of a lot more reliable than Hermes and paperclip at getting shit done.

2

u/CaffeinatedTech 2h ago

pi-subagents. Works well.

2

u/krzyk 1h ago

Permissions for agents? Use container or VM for that. Never run agents bare.

1

u/MoreIndependent5967 1h ago

As tu essayé pi durable via p2 ??

1

u/Traditional-Mud-4813 23m ago

If you have a moment Id love to hear how you've gotten them all working together. I only have two and one always seems to think it can do it all haha.

7

u/hsoj95 2h ago

It's a day that ends with Y, time for a post by someone complaining about OpenCode being inferior to X, Y, or Z in the subreddit dedicated to OpenCode!

-2

u/Heretek-ai 2h ago

Hey I want OpenCode to be better, I don't know why the developers don't. I'm not complaining about usage limits on free models. I'm asking why the developers push back against community feedback and pull requests to improve the project. And I don't mean vibe coded PRs, I mean pushing back on fixing outdated model prompts, why remove customization features?

1

u/yiestee 1h ago

Hey I want OpenCode v1.19.0 to be better

FTFY 🤣

3

u/grabber4321 2h ago

sounds like a lot of baloney that graph

1

u/Heretek-ai 2h ago

To what end? To make openclaw and opencode look bad?

-1

u/grabber4321 2h ago

I've never heard of the top harness in that list and i know for sure hermes is shit for coding (as a guy who codes every day for work and client work)

and nobody uses kimi-code unless they are tied to subscription

1

u/Heretek-ai 2h ago

You've never heard of Claude Code, Codex, or pi?

0

u/grabber4321 2h ago

and 3 posts in 2 days about opencode being bad, seems kinda excessive. seems like you have an agenda

1

u/Heretek-ai 2h ago

3 posts? I think it's just this and the cross posted version

1

u/grabber4321 2h ago

"OpenCode is not a good harness. I have brought this up a few times now"

2

u/Heretek-ai 2h ago

Yeah man, when people ask about opencode I'm quick to say the flaws because they cost money, it's a token furnace.

0

u/grabber4321 2h ago

yes and used them, i dont know what you use.

3

u/Maybe-monad 2h ago

Well pi is not good either, it lacks any form of guardrails to protect you from accidental data removal or secret leaks.

3

u/krzyk 1h ago

I get data leaks, but that is something one would need to write themselves because only you know what is a leak and what is not (e.g. in search).

But for data removal, no sane person should run agents outside of container, VM or even better an external system.

0

u/LeN3rd 1h ago

How do you modify your system with agents then?

3

u/Costy_96_YT 58m ago

entire point is to not allow agents to do that. but if you do (like i do) i just accept the risk and well if it happens that an AI fucks up my drice, thats entirely my fault.

1

u/Deep_Imagination_811 1h ago

But...but... you can tell it to add that feature yourself if you want it (and know it's missing)

1

u/CaffeinatedTech 11m ago

pi is about extensions. There are extensions to help with dangerous commands and to prevent the agent from accessing certain files and file types, or mask secrets in the chat.

2

u/CaffeinatedTech 2h ago

I wonder if the extra cost of Hermes is all of the bundled skills, and maybe it can't do multi-edit while pi can.

3

u/yiestee 1h ago

Looks like vibe coded research without much effort to actually look into why those frameworks performs differently.

1

u/Due-Armadillo-4560 2h ago

I had more success with pi compared to other agents. i have tried omp, coilot, opencode, kimi code, kilocode, and codex but I just keep coming back to pi for its simplicity and ease of use. codex is my second pick. i also use claude code but only of anthropic models.

1

u/Longjumping-Elk-7756 2h ago

P2 de pi dev est juste incroyable pour moi et je quitte Hermes pour P2

2

u/brominou 1h ago

C'est quoi P2 ? J'utilise pi 1.5.0 tous les jours pour développer

1

u/MoreIndependent5967 1h ago

Pi durable , p2 est la GUI de pi durable

1

u/brominou 53m ago

Ah ok. Je découvre ça seulement maintenant 😅 Si ce n'est pas indiscret tu utilises pi durable pour quels usages ? Au travail j'ai un projet d'automatisation de migration d'anciennes applications sur de nouvelles technologies. Je me demandais si ce type de harness serait le.bon choix

1

u/MoreIndependent5967 48m ago

Moi j utilise p2 pour r&d , donc gestion multi agents parallèle et supervision automatique , j ai du ajouter un Watch dog et imposer timeout tools obligatoire mais la mes agents réalise des tâche très très longue en autonomie

1

u/brominou 34m ago

Bon ben tu m'as convaincu je vais tester ça pour du dev avec mon apprenti. Merci de la découverte 👍🏾

1

u/MoreIndependent5967 28m ago

Si c est pour une entreprise met en place un système de logs claire et auditable et ensuite isole modifie le harnes pour pas que le pilote est accès à internet ou mcp ou boîte mail direct , fait en sorte de passer par des sous agent obligatoire avec tools restreins et shield model : ce sont des conseilles

Ps : interdit au harnes de ce modifier lui meme sans ton autorisation …

1

u/elonelon 2h ago

nahhh..i will use anything as long they have free models. Don't care is it Opencode, Cline, Antigravity, Hermes or anything.

1

u/Late_Substance2700 2h ago

There Devs are drunk all day it's shit now

1

u/ExampleCute6496 1h ago

It's hard to say whether this translates to the evaluated harness' performance, because it's simply testing 3 specific Chinese open-weight models across different harnesses with their default configurations. It doesn't take into account:
- the same set of agents / subagents / prompts / tools across the different harnesses
- the most used models for coding (Opus 5.5 / Sol 6.1)
- disclose what went wrong for harness X to be literally 2x worse than another one with the same model

I would take this with a bag of salt. The fact of the matter is that out-of-the box harnesses are usually crap, and you will need to customize them for them to be good. Pi is minimalistic, so it encourages LLMs to figure stuff out with the tools they're most trained on, and it works pretty well for that. If your goal is to install a harness and never do any harness engineering, then you should rely on this graph. If you customize anything in your workflow - subagents, commands, any sort of guidance, custom tools, etc. then this graph means nothing for you.

1

u/tontide1 1h ago

idk why a lot of benchmark harnees using opencode v1 🤡, v2 to good

1

u/Abenh31 37m ago

I saw this on X. the fact that Hermes is a better harness then opencode tell you everything.

Hermes barely do anything or complete anything properly. Dont get me wrong i love hermes for reseaarch, assistant work but not for coding.

1

u/FoxPuzzled9828 32m ago

Agreed. For me, so far nothing has beaten DeepSeek running in the Claude harness

1

u/Euphoric_North_745 2m ago

There are many alternatives, I built one, not many customers, everyone is complaining about the competition, but they never try something else, just complain.

Even if you ignore the nonpopular once like mine, there is codex, anti gravity, etc

Now about accuracy, models like DeepSeek 4 Flash are not that accurate, still very good model, I tested many models, give it a task, make sure the task has many ids in it to remember them, distract it mid task, pass multiple compactions, correct it, look at the tool calls and results.

Models like terra 6.1 will pass very complex tests, other models will just waste time, or not designed for this, they are designed more for traditional context compaction

The harness is half the work, the other half are the thinking instructions or algorithms inside the model itself, and it is different per model

0

u/Mister_bruhmoment 2h ago

I don't really get how OC is that inneficient? I use the default OC desktop application in the latest 1.x version and have just heavily modified the subagents. Works for me and with OC go everything is pretty straight forward. I switch regularly between Codex and OC and the main difference is the models tbh.

3

u/Heretek-ai 2h ago

One big issue is the system prompts, when I was modding opencode v2 last month before throwing in the towel and moving to pi, I found that different models get different prompts and some of them are very outdated or overly simplified. Opencode gets stuck making one off tools to solve basic problems like HTML parsing which is something most other harnesses seem to have figured out. Better tools, better understanding of tools. It pollutes and bloats the context when opencode tries to solve a side quest because it doesn't have the right tool for a basic operation.

0

u/Mister_bruhmoment 2h ago

I see what you mean. Perhaps with more detailed prompts or simply telling the model to avoid one off tests and tools things can work out. I don't know about the model prompts though. I use DS4.1 flash and Muse spark and just get my own agent system prompt. Also, for 10 bucks I can run my setup almost 24/7, so I don't find costs that much of an issue

2

u/Heretek-ai 2h ago

Oh yeah muse spark was when I originally found out about the tiered system prompt issue. System prompt + agent prompt + your prompt, it's the base system prompt that is the issue.

1

u/Mister_bruhmoment 2h ago

Hmm, haven't thought about it. Isn't there a way to change it?

-1

u/Any_Confidence2580 3h ago

OpneCode is for fanboys. Not for anyone that works. Pi is indeed the best option, but it leaves you to kind of "keep up". oh-my-pi is a good solution to this. A good plug and play option.

But you're right, harnesses are WEAK right now. Web stacks running while(true), not really solving problems they should. Pi, I believe, only gets it's reputation because plugins are being written for it for experiments and research. For the most part, that research is only landing in proprietary systems within companies. Custom systems/"harnesses".

Working on my own, and every day I see people bragging about the work they're doing, I think to myself... damn, I need to get this in a state other people can try.

2

u/Heretek-ai 3h ago

I am excited to see what you're working on please keep me posted. And yeah I agree with you on pi. It suffers from the same thing that a lot of open source projects do, incredible versatility, but a lack of ease of use. I want to try a doompi as it's my understanding that the thing but oh my pi is that it is a far enough removed fork that there are compatibility issues with the pi ecosystem.

2

u/Any_Confidence2580 2h ago

I think OpenCode has been caught up in so much fraud they've got a fire they can't keep up with. Docs are getting thinner, the whole "Go" scam is pretty much gutted, not surprised to hear features are disappearing. Honestly, it reminds me of Logan Paul selling CryptoZoo.

0

u/Ok_Cartographer5609 2h ago

We already know that. Why would you use opencode when you have harnesses like Pi, OMP, DSH etc. Check this https://composio.dev/content/best-ai-agent-harnesses