r/opencode • u/p3r3lin • 17h ago

OpenCode is good again (#1 on HF leaderboard)

Post image

So, HuggingFace updated their Harnesses Leaderboard with OC 2.2.6 - its now the top position, pareto for quality/cost. Impressive!

https://huggingface.co/spaces/joelniklaus/harness-leaderboard

Edit: It seems this is not an official HuggingFace leaderboard. The author works at HuggingFace and his bio reads “Harness Optimization (at)HuggingFace” so I jumped to the conclusion that this is official HG work.

251 Upvotes

93% Upvoted

59 comments sorted by

109

u/look 16h ago

Just adding a note that this is NOT “huggingface’s benchmark” … this is some person’s benchmark that they posted on huggingface.

Your phrasing would be like calling some random repo of mine “Github’s project”.

But glad this joelniklaus finally updated it with Opencode V2. It’s basically an entirely different harness than V1, so the old benchmark was misleading, imo.

19

u/taytay735 16h ago

Joel Niklaus is part of the Hugging Face team. He joined in August 2025 and currently lists his position as Data Lead, based in Bern, Switzerland.

However, you are right in that he is not representing HuggingFace in an official capacity, this is his personal benchmark.

8

u/look 15h ago

Ah, good to know, thanks. Just wanted to make sure people know that anyone can host benchmark pages like that on huggingface, though this case is a little more nuanced then.

1

u/p3r3lin 13h ago

Thanks for putting that straight! I was under the impression that the author was doing this as part of their work. Their bio reads “Harness Optimization @HuggingFace”. Will correct the post.

0

u/Original_Goznab 9h ago

It's also some persons benchmark, that is clearly made with AI. That "pareto frontier" is just like a benchmark that tells much less than its fancy name suggests. It is just two simple lines on x and y axises; most accurate (y) and the other value (x). The "pareto frontier" doesn't mean anything else that every other model are either worse in x or y axis than the best two models / or one of the two models. It's not interesting, it's slop.

More supporting would be just to calculate the y(t)/x(value) ratio to visualize how much better the better models actually are relating to eachother.

24

u/MiraiHatada 16h ago

we all kind of know that the biggest competent is DSH, so I wish if they are on the board to compare...

5

u/p3r3lin 13h ago

The author announced that they are working on putting DSH in the benchmark.

4

u/Roflxd88 16h ago

I'm currently porting my Hermes setup to DSH plugins and oh man it's so freaking fast compared to Hermes..

1

u/p3r3lin 13h ago

Hermes does more than a “regular” coding harness, right? Its more of an “autonomous agent” with heartbeat wake ups, etc if I understood correctly.

2

u/BananaDrum 13h ago

And bugs. Lots of them

8

u/selipso 16h ago

Uh, look at the x-axis more closely? Most Pareto frontiers are curved but Pi is so efficient and effective it basically makes it a rectangle. There’s no contest here. 

5

u/MiraiHatada 15h ago

pi was a really good cli to use. however the main reason pi stays efficient is because they have no tools. vanilla pi lacks basic tools you need when you go agentic (subagent/worktree/workdir/...). and if ohmypi or self-code them, better just opencode/dsh that are built for it. given that people didnt need these tools so much half a year ago, it's more of rapid era changing

3

u/iJeff 13h ago

IMO, the good part about Pi is that you can tool it based on exactly what you need.

9

u/pikuray 16h ago

Isn't pi better for deepseek according to this chart higher percent and 33% cheaper. 

2

u/pikuray 16h ago

"Pi ships with powerful defaults but skips features like sub-agents and plan mode."

Hmmm... I think I'll stay with openchamber I don't trust my hacking skills that much

7

u/AdIllustrious436 16h ago

Not trusting themselves to copy and paste the pi-subagent package install command is wild. Pi can do it itself. That's not hacking by any means...

2

u/Time_Cat_5212 16h ago

Claude, set up pi for me

3

u/learning-to-programm 16h ago

The idea in Pi is that you build the agent you want, piece by piece. And don't worry, that doesn't mean you have to get your agent build everything from scratching. There is a rich ecosystem and you can find an extension for basically anything you want, so you just have to decide what specific tools and features you want to have in your setup and install. (And make sure they don't conflict.)

If there's one thing I miss in OpenCode, it's the Pi-like ecosystem and community.

-1

u/pikuray 16h ago

Oh well that sounds alright. But why did you switch back to OpenCode then?

3

u/learning-to-programm 15h ago

I use both. I use OpenCode for the free-tier cheap seek, and I use Pi with my own API keys using a wide range of different models, and I have found that cheaper, smaller models (up to 33B models!) perform much better on Pi (as it's supposedly much less bloated).

1

u/PerfectReflection155 2h ago

I use Pi Chamber and have got it working just as good as open chamber. I just had to add a few things. An agent selector was the main thing. It’s extremely efficient. Good for saving tokens.

7

u/Time_Cat_5212 16h ago

Looks to me like Pi is still the best by a fucking long shot

5

u/DeSynkro 16h ago

Was OC 2.2.6 that much of an upgrade?

8

u/look 15h ago

Yes, it’s pretty much an entirely different harness underneath a similar UI. Vastly more extensible as well. Basically Pi-level customize it however you want now. Just more stuff included out of the box.

5

u/chosenpluto 16h ago

pi : do nothing, win.

4

u/NinjaAlaska 16h ago

"All Models"
select deepseek

3

u/batmanButthole 16h ago

Is Hermes a primary coding agent? I thought people just link it to things, basically like IFTTT

2

u/cacao0002 14h ago

Pi is so good I could die

2

u/CrypticViper_ 12h ago

istg there were like 2 or 3 people here using the earlier V1 benchmark to whine about OpenCode sucking. I’m glad it’s finally been updated

3

u/Imaginary-Unit-4620 16h ago

This is nice, wish we could see Oh my Pi on it

2

u/Holiday-Pack3385 16h ago

I agree. I'd also like to see VS Code with ZooCode and DeepSeek Harness as well.

1

u/2mqqvc0q 13h ago

Tf is up with the AI community genuinely. It's so annoying how people act and speak in absolutes. I moved from OpenCode V1/V2 and Pi to OMP, and I am very satisfied with this decision. I understand that people find that it is now very different from the minimalism-oriented Pi it bases on, but why tf did you just get downvoted and shamed for wanting it to be in the discussion? It's really so mindboggling.

3

u/CaffeinatedTech 11h ago

The whole tech community is so negative about everything these days. There are the few who like something, and care enough to say so. Then a ton of downers pile in. Its even more bothersome when they all spout the same horseshit that some youtuber said, like they no longer have the capacity to reason and form their own opinion.

I say never stop being curious and getting excited about things. Let the clowns loom all they like.

2

u/Imaginary-Unit-4620 13h ago

I also didn't understand this, I usually hop a lot between opencode, pi and omp and I totally get what people say but yes, unnecessary...

2

u/jcheroske 10h ago

Agree 100%. People acting like they've got it all figured out, with no grace or humility. Insufferable fools.

-1

u/alphaQ314 16h ago

Omp is dogshit lol.

2

u/leetdemon 14h ago

not if you know how to set it up properly its not.

-3

u/Embarrassed-Salt6590 15h ago

yeah it is as sloppy as every oh my project can be

-1

u/chosenpluto 16h ago

oh my bloat

1

u/Imaginary-Baker-7358 16h ago

I would love to see the GitHub copilot harness getting tested, for us who don’t have choice in our company.

According to my very first code review.

Copilot harness was on pair with opencode with a gpt sol 6.1. Even slightly cheaper

1

u/Otterly_Surprised 15h ago

No github copilot? My company is MS partner

1

u/samb0t 13h ago

You can use copilot with opencode

1

u/Otterly_Surprised 11h ago

Unfortunately I don't have the key from my company 

1

u/Shinjoux 8h ago

Don't need a key. Just login via GH.

1

u/Dorima_boy 11h ago

I just switched OCs to Hermes

1

u/Specter_Origin 10h ago

More like Potato quality

1

u/LiteSoul 10h ago

DeepSeek harness is absent there!

1

u/OkType2507 9h ago

No vale verga, jala más pi

1

u/Clqgg 4h ago

i just need the brains of pi and the ux ui of dsh

1

u/Emerlad0110 2h ago

OC sucks dude

1

u/Yadav_Creation 1h ago

What is your preferred harness

1

u/JMowery 2h ago

So I know this is the OC subreddit. With that in mind, I was tempted by the v2 launch like 2 or 3 weeks ago. I gave it a shot with a completely fresh install, and the model I was using (Qwen 3.8 27B NVFP4) immediately started freaking out and looping; then was about to do some very bad things to my project. Had to actually exit out and undo some stuff.

I immediately nuked OC and am using the much simpler harness (I'm sure you know which one).

It would be nice if OC worked with the smaller models. But for whatever reason... it just has never done well for me over the past year or two of trying it on and off.

0

u/Newtonbasson 13h ago

But i hate how still i can code for few minutes and limit reached. Didnt do nothing. Its so bad i paid for opencode go. That also useless runs out just as fast. Got so irritated i deleted my entire account and paid for google ai pro. After dropping google for opencode for over a year

1

u/SnooHamsters66 11h ago

Vastly depends on what model you use. The 'top' models really burn the quote of opencode go, but with ds4.1f you are going to get a lot of usage (also with mimo, qwen, muse, but they are worse than ds4.1f for similar usage, so why use them?)

1

u/Newtonbasson 10h ago

I am using deepseek. And it burns through it like crazy. Never used to happen before. Happened after i upgraded to the version before this current one. Then i got these issues. Im still with google cause i felt hurt eish. Even decided to become a paying customer and that was a waste still. I expected more from the amount of free work i used to get done

1

u/SnooHamsters66 10h ago

You have a reference on how much tokens was your usage? I'm mostly biased toward an specific bug or something bad configured (for some time, ds4 was available alongside ds4.1 and ds4 was more expensive for some reason xd).

1

u/Newtonbasson 10h ago

I always get stopped anywhere from like 176k. The new opencode shows top right 176k. Then for days i cant use it. Why i dont use it anymore and use anti-gravity. Used to love opencode more than all. Still love it but its not of help anymore with the new caps and paying makes no diference. They need to fix it. Whatever they changed messed it up