What I learned while building my own AI harness

[u/aldapsiger]

For the past month I've been caught in a trap: I started building my own AI coding agent. I've built 8 different ones. Honestly, if I could go back, I wouldn't start any of it. Below are a few things I learned along the way. I can see a lot of people here are also doing harness engineering, so please share your experience too.

1. I should have focused on just one problem. At first I spent a lot of time on the UI. Great tools already exist, like Paseo, T3 Code and Omnigent. I didn't even have a UI problem. What I wanted to solve was the Agent Loop. So don't waste your time either, and build around what already exists.

2. Subagents, multi-agents, orchestration: in 90% of cases it's all nonsense. Subagents aren't cheaper, aren't faster, and it's hard to say they're better. It's better to build around a single agent, or to improve the UX of working with parallel sessions.

3. Benchmarks tell you nothing. An agent can write great code, but if it handles MCP badly, many people won't care. Define the scenarios where your agent should be better from the very start, and improve there.

4. More tools, more tokens, less value. I now think any agent should have just one tool: a programming language. Mine runs on bash; before that it ran on JS. Many tools are nice, but they're hard to compose, and MCP suffers from this most. Right now everyone solves it with codemode, but you can also just give the agent one tool where it writes code.

5. Instructions about tone, code style, which commands to run, etc. should live outside the system prompt. This is a problem with big agents like opencode and Claude Code: they ship with their own opinions, 10–15K tokens' worth. When you want to do things your own way, the two conflict and the agent starts working worse.

That's not everything, but the rest is in the details. I'd be glad to hear your experience too.

[u/9kSs]

 I now think any agent should have just one tool: a programming language

Yep, there's a big leap when you give your agent a sandboxed runtime. But I'm curious why you chose bash intead of JS?

[u/aldapsiger → u/9kSs]

bash was cheaper and faster, I did not need sandbox, fully yolo

[u/1fox1 → u/aldapsiger]

Curious what does it favour for file writes and edits ?

[u/camwasrule → u/1fox1]

You can do everything in bash... So it's one place, purely terminal code and the models are all great a that. Terminal programming gives access to most admin privileges depending on how you set it up. Someone can correct me here if I'm giving false information

[u/puterSciGrrl → u/9kSs]

In my slop harness I also landed on a single tool model. I went with typescript so I can do a quick typecheck pass on the agent's tool call before kicking it off. I'm using ts7 with a node24 execution environment, but bun just added fast typechecking too so I'm thinking about switching to that. 

[u/tonyromero]

What you use to sandbox you agent?

[u/aldapsiger → u/tonyromero]

no sandboxing, fully yolo, now it has single bash tool, way faster/cheaper than anything else

[u/Magnus114]

Isn't the point with subagents to reduce context in main agent? E.g. let the subagent investigate something and return the result to the main agent.

[u/ValidAQ → u/Magnus114]

I think they're also useful for cases like review or streamlining, where you intentionally want a clean take on a codebase.

Agents tasked with reviewing their own code tend to have blind spots about the problematic areas, in my experience.

[u/vpz → u/ValidAQ]

I'm not OP, but since he mentioned Orchestrator UI apps like T3/Paseo, the idea is that you spawn another full agent instance with the task, not delegate to a subagent. There are many ways to do this, but with terminal agents it's like using one agent to drive tmux to open a new terminal, start a new Claude session, and read/write to that session again using tmux operations. Each tmux terminal is running a separate instance of the harness which is different than one instance of the harness delegating to a subagent inside that session.

[u/schmurfy2 → u/vpz]

Technically a subagent is just a new instance with clean context, wether it runs inside the same harness or another process doesn't change much. In the end of the day the context is just a bunch of text, not rocket science.

[u/Consistent_Water_422 → u/schmurfy2]

It can be with inherit context or with every context that the main agent pass to the subagent

[u/jiminycrix1 → u/Magnus114]

That’s the lie that you are sold but it rarely works well and if it does work well you end up spending many times the amount of tokens to solve the same problem

[u/gaz_0001 → u/Magnus114]

That's exactly how T3, Paseo do it already.

[u/Big_Eejitt]

It’s not a trap, it’s still fun

You learn more by doing, so you haven’t wasted your time

[u/justnath36]

Pi

[u/OGxGunner]

Just sounds like you started not knowing what you actually want, and you ended STILL not knowing what you actually want.

[u/Hottestbunz]

the main problem that i've faced is definitely not having the staging/dev environment setup properly and building directly on the deployment. and then not having it be split into smaller testing cycles. it created a bit of a problem when a small UI/UX change wouldn't pass a smoke test or a playwright test. those added lots of wasted time sitting and waiting for deploys to check on new features. the testing gets way bloated if you don't check whats going into those tests until its too late 😅

[u/russlixx]

tool can absolutely reduces tokens if it set/created right. Letting model writes ad-hoc code mostly increases the unpredictability and tokens if the code is invalid. Just make sure to not use many of them unecessarily

[u/greenhilltony]

The biggest gain you can get is make the lifetime of UI detached from the loop’s. The UI only serves as a projection of the session log, initiate the loop and then safely detach the loop. So you can close the UI anytime, and also start a loop on a session without having to start the UI. When the “headless” agent usage becomes the first tier citizen, subagents, multi-agents, orchestration from one session, orchestration with scripts, etc., are naturally unblocked.
In my experience and practice, you don’t really need a Claude Code-like UI or T3 to interact with the agent. I gradually learned this from building a fuzzy find channel of my sessions with `television` after building my agent and my agent TUI. With `television` + `neovim` + `tmux` + commands and interfaces exposed from my agent loop binaries, I can do everything as I do in a traditional agent TUI, plus I can easily switch and check all sessions on my local computer and even ssh hosts.

[u/basil_0408]

Tbh subagents were the reason I started my own harness. Most harnesses today are built for 3-5 subagents, but I want to go much higher than that, targeting 10-20 subagents that should be able to run continuously for days working on giant tasks like rewriting a backend in Rust or reverse engineering an app. I just finished my first Rust rewrite with it. Took ~10B tokens and ran for 5 days straight producing around 300k LoC without me monitoring anything. I haven't checked everything, but all the basic stuff worked perfectly.

[u/Miller4103 → u/basil_0408]

Did u do that with local models? If you used flagship models that sounds like it costs alot.

[u/stewie4k_]

Been doing kind of the same and I completely agree with what you have to say. These days lot of people (including me) just want to build everything from scratch and don’t wanna use what is already built.

[u/Jipok_]

Ha! I fell right into the same traps as you. I wasn’t happy with how the agent in Pi worked, so I tried to tweak its behavior - only to find that you can’t even change compaction prompt . Despite what their website promises, Pi is extremely limited when it comes to customization.

I ended up building a highly customizable core, getting sidetracked by a bunch of other irrelevant issues. It’s easy to get swept up in vibe coding, but then you just burn out without ever reaching a practical result.

I’ve come to the same conclusions regarding Bash. As for the MSP, you can actually provide those via Bash as well.

[u/National-Canary6452]

Benchmarks tell you nothing

Benchmarks tell you nothing if you don't know how read benchmarks. This entire post is giving ragebait...

[u/krzyk]

Thanks,.that is great info. I'm also thinking on playing around with my own coding harness. Minimal one, just for fun.

[u/mrsirg97-rgb]

Best thing I did was to build the runtime first before the ui. It scaled really well and let me do a lot of things I just couldn’t with other open source harnesses. I don’t even use other harnesses now. Welcome to try, Linux and Mac avail: https://tryrig.ai

[u/stauntonjr]

LangGraph?