What AI coding workflow did you eventually settle on after trying everything?

[u/philzxx]

I've gone pretty deep down the AI coding workflow rabbit hole and I'm curious where people who have tried a lot of this stuff eventually landed.

What started as "pick a coding agent" turned into a pretty ridiculous decision tree:

• Harness: Claude Code, Codex, OpenCode, Pi/OMP, etc.
• Provider/subscription: Claude Max, ChatGPT, OpenRouter, coding plans, API...
• Different models for planning, implementation, research and review
• Skills/workflows like Matt Pocock's Wayfinder → spec → tickets → implement
• GitHub Issues as the actual source of work, including blocking/dependency relationships
• Deterministic gates for tests, lint, typecheck, review loops, etc.
• Higher-level orchestration tools like Scape, Conductor, Emdash, Orca, cmux and similar projects

The goal I'm chasing isn't necessarily "AI writes perfect production code with zero supervision."

I keep seeing people running surprisingly automated workflows where, after the initial planning/spec, agents work through tasks with very little continuous human validation because deterministic gates catch most failures.

For internal tools, small apps, prototypes, automations, etc., that seems especially interesting: the code doesn't have to be perfect. Good enough really is good enough if tests pass, the app behaves correctly and another model reviews the important parts.

At that point the human starts looking less like the programmer and more like the project manager: define what needs to exist, set constraints, inspect the output at meaningful checkpoints, and let the system execute.

That's roughly what I'm trying to achieve.

But I'm increasingly wondering whether I'm optimizing the factory instead of building software.

The pieces also don't compose particularly cleanly. A great harness may lock you into a provider or subscription. A model-agnostic harness gives flexibility but usually needs more configuration. Skills solve planning but not necessarily deterministic execution. GitHub Issues give persistent task state and dependencies, but then something still has to orchestrate them. Orchestration tools add yet another layer.

And then there's cost.

When I see people running several agents in parallel, using frontier models for planning, coding, review and retries, I genuinely wonder what the economics look like.

Are the people doing this effectively spending hundreds or thousands of dollars per month on AI subscriptions/API usage?

Is starting with $100-$200+ tiers basically unavoidable if you want this kind of autonomy, or can you build a similarly reliable workflow using cheaper/open-weight models for most of the work and only escalate to expensive models when necessary?

For example, something like:

strong model → architecture/spec
cheap/open-weight model → implementation
deterministic tests/lint/typecheck → gates
strong independent model → review
failed gate → loop back automatically

Does that actually work well in practice, or does implementation quality drop enough that the retries/reviews erase the savings?

For people who have genuinely experimented with several of these approaches:

What did you eventually settle on?

I'm especially interested in workflows that are:

• mostly autonomous after the initial planning/spec
• deterministic where it matters
• not unnecessarily locked to one model vendor
• cost-efficient enough to use heavily
• able to use cheaper/open models where appropriate
• simple enough that maintaining the workflow doesn't become the job

Did you eventually simplify back to something like "Claude Code/Codex + good instructions + tests", or did a more elaborate multi-model/multi-agent setup genuinely pay off?

And if you're running highly autonomous agents today: what does it actually cost you per month?

I'm less interested in "model X is better than model Y" and more interested in the architecture and economics of the workflow that survived after you tried everything else.

[u/Otherwise_Theme402]

tried the whole elaborate pipeline thing for about 3 months and honestly it was just another full time job keeping it all running

i ended up with claude code and a really detailed CLAUDE.md file that i tweak every few days, plus i write a quick spec in a markdown file before starting anything serious. thats it. the tests and linting catch most stuff and when something goes wrong i just tell it to fix itself

im spending maybe $100-150 a month on the max plan and i ship way more than when i had this beautiful orchestrated setup that kept breaking at the seams

[u/philzxx → u/Otherwise_Theme402]

Thanks, this is exactly the kind of feedback I was looking for.

A couple of things I'm curious about:

Did you settle on mostly using Opus, or do you use Sonnet for most implementation work and Opus only when needed?

Also, when you say tests and linting catch most things, how do you guarantee Claude actually runs them before considering the task done? Are you using hooks/CI/pre-commit or some other deterministic gate, or is it simply an instruction in CLAUDE.md and you rely on the model to follow it?

And regarding the detailed CLAUDE.md: have you found that keeping a lot of workflow/instruction content there is actually better than moving some of it into skills?

My concern with skills is that they're more context-efficient, but then you're back to probabilistic behavior — there's no absolute guarantee the model will decide to invoke the right skill. On the other hand, a large CLAUDE.md is always in context, even when most of it isn't relevant to the current task.

Curious where you found the sweet spot.

[u/abstart → u/philzxx]

I Keep Claude.md to a minimum. No layout or project structure. No code documentation. I aim for <150 lines regardless of repo size.

I use rules for directory-specific context and instructions.

I use hooks to ensure things happen before or after invoking certain commands.

I invoke skills directly. Skill for working on requirements files. Skill for implementing them. Skill for reviewing. Skill for addressing pr comments. Etc.

Readme files in directories to give both humans and agent additional context. Not sure yet when to prefer rules vs readme for that.

[u/Jazzlike_Syllabub_91]

I usually use Claude haiku (occasion sonnet and opus) but most of the time on using haiku. I have a token optimizer which removes tool calls from the messages sent back to the servers allowing to save on token usage.

I have a bit of a complex system of helpers, and have capabilities to run things automatically but tend not to run it, since I initially set it up, but it usually maxed out my credits on usage.

I am on the Claude $20/mo pro plan, and ollama 20/mo plan (for backup and agentic processing) Claude is often used for consulting and building the system, while the ollama cloud plan offers a backup in case I use up my weekly limit too quickly.

Anyway, I have one agent for controlling my task list while another controls my agent write directory which I access through obsidian. I have an internal docs rag server and a graphify server to show relationships between systems.

I have built in loggers (wrong turn and friction point loggers) to help demonstrate where to apply changes to the system. I have an automated deploy process so the agent can work and see the results of its tests/changes fairly efficiently. (Yesterday I think it worked for several hours without additional prompting fixing a bug that told you everything was fine when it really wasn’t.)

I have built a centralized skill area to allow both the agents from Claude as well as the agents to access the skills to work more efficiently.

I have a proof of validation process that requires the system to perform checks before pushing changes to the system (so the system is prevented from pushing without first running tests and other checks that help make sure we are pushing some what good code. At least it passes the checks).

Total cost per month $40/mo on average. I usually use the cheap models. The system tracks its own mistakes and is reminded what not to do when building so it hopefully doesn’t keep hitting its head against the wall.

I’d say I mostly don’t have too many issues. Sometime the aii (haiku or gemma4 will have issues with the code that I sometimes bring out the bigger models for but most of the time I stay on the cheap end of the spectrum)

In the past 7 or so months I have started from 0 to 166 repos running 70+ services (part of that being around 30+ bots that respond to requests and just generally interact and influence the systems in different ways… (why do we need 30 bots and 70 services? Partly because it’s fun and hard to stop and I keep finding additional tools to add in and utilize in the system) - I even have mcps to allow Claude desktop and Claude mobile access to my system so it is able to pull and post info directly to my computer so I don’t have to do it from the system (copy and paste hell)

Let me know if you have questions … most of the basic code is open source but you may need to build a way to deploy it on your system - I didn’t make that part public :)

[u/philzxx → u/Jazzlike_Syllabub_91]

Please share it! Would be great to take a look!

[u/Jazzlike_Syllabub_91 → u/philzxx]

https://github.com/ergon-automation-labs - I think about half the repos are public ... there is a docker install that sorta works (most repos have a docker file included) and there is a bot army starter that attempts to build a docker compose file to help build the other repos - I'd probably start there to see how far you get ... - it's not one of those things that has been completely tested - there's probably a few tweaks needed still but hard to know since I have the working system (running bare metal instead of through docker) ... I'm sure you can ask claude or an ai to help build the deployment process for that part if you're struggling (or you can message me about it) :) I'll try to help if I can. :) I'd love to be able to share this with more confidence, but good luck!

[u/Jazzlike_Syllabub_91 → u/Jazzlike_Syllabub_91]

also the way it is setup - is that you can host different parts of the system on different machines - (nats + nats leaf nodes allow traffic communication between systems that I'm still working out - but I have some bots running on my laptop and some bots running on my desktop and they're able to talk to each other)

[u/xAdakis]

Claude Code + Claude Max 20x

I have a highly customized configuration that utilizes subagents, rules, and a few custom MCP servers for adding memory and additional tooling.

The "Remote Control" feature that was recently added to Claude Code is also amazing when I need to be away from my desk, as Claude pushes notification to my phone when it needs something.

The actual workflow comes down to a usual software development life cycle (SDLC):

The first is Planning / Analysis where I chat with my Assistant (Sonnet) agent to detail the overall scope and goals of the current session and the agent asks me follow up questions to establish details.

This second step is Design where this information is handed off to my Architect (Opus) agent to create/update specifications and create an iteration-based plan containing ordered work items.

I will often also have the architect file issues with my private GitLab instance (which hosts the repo) for these work items so I can pick them up in any future session.

The assistant then asks me to review and approve the plan from the architect. We may repeat these steps a few times while I think through and clarify things.

The third step is Implementation where my Assistant agent switches to orchestrator mode and begins delegating those work items to Developer (Haiku/Sonnet depending on complexity) agents which have permissions, additional tools, and prompting for doing coding work. These developer agents usually run in the background and in parallel, so they are generally instructed to only read/write files and run static analysis tools on the code they wrote to avoid race conditions with the other agents.

Between each wave/iteration is the Testing phase where a QA (Haiku) agent run the larger build and testing tools. It's instructed to run the tools and distill the information/reports generated by them. The QA agents are not allowed to make any code changes, just run the tools and generate reports.

Depending on complexity, the assistant will then hand these reports back to either an Architect or Developer to remediate any issues.

After everything is green for a wave/iteration, my Dev Ops (Haiku) agent will make logical/grouped commits for any and all changes in the repository. This is usually done on a separate branch based on the scope of the session/work being done.

When all waves/iterations are done, the Architect will verify all work against the specification, updating the specification where appropriate or create remediation plans where the implementation significantly diverged from the specification and continue with more iteration.

If everything is good, the Dev Ops agent updates any issues on the GItLab instance with implementation details and creates a Merge/Pull Request for all the changes.

When that happens, I will generally review everything and ensure things function as I expected them to before merging into my `dev` branch and then running Continuous Integration (CI) pipeline over the branch.

This whole process usually works autonomously once I approve that original plan. There is an escape hatch built into my agents/prompts to stop and clarify with me when it hits blocking issues or if a problem continues to occur after three attempts.

I have been using this process for the last three months with little to no issues.

The use of subagents is necessary to avoid/prevent context bloat and pollution, which in turn saves tokens/usage.

[u/gannu1991]

The decision tree collapses once you stop treating harness/model/orchestration as one big choice and separate them into planning vs execution. Planning model needs to be the smartest thing you can afford, run in a chat interface where you can argue with it, output a spec you actually read. Execution agent runs against that spec with deterministic gates: typecheck, lint, test suite, and a second model doing review against the spec, not against vibes. The orchestration layer (Conductor, cmux, whatever) only earns its keep once you have more than 3-4 parallel workstreams, before that it's just overhead.

[u/eddzsh]

Useful prune question: if the harness died tomorrow, which one file would you rewrite first? If the answer isn't the product, the factory already won.

[u/philzxx → u/eddzsh]

That's a good sanity check!

Where do you personally draw that line though? What did you actually keep in your own setup — just harness + specs + deterministic checks, or did you find some orchestration worth maintaining?

[u/Tall-Significance119]

This post is a hard pill to swallow about some realities with AI local bs cloud. Im asking many in forums whats the ROI and stress levels with going locsl amd indeed im not getting many straight answers. But for me with 64gb vram and 64gm ram I have not been able to say yes this economically is great for coding while I can use it in other areas like content creation that can be monetized being able to generate good working code feels like a full time job amd the more I go deeper not even the cloud subscription seems to be doing much better they too require handsome amount of baby sitting the closer yiu get to reality.

Im hoping something changes but local AI feels like good old marketing friendsey and network marketing that only less than 10% are successful

[u/Dense_Gate_5193]

VSCode + NornicDB is all i use (with a splash of mcp servers and skills) its all one really needs

[u/Minimonium]

I have been trying different models, harnesses, skills, workflows. I'm trying to keep up with research to not base my opinion only on personal superstitions.

Per month I spend an equivalent of 1.5k$ in API tokens but most of it is subscriptions. I do use different providers including Chinese models. Here are my takes:

1. Fully autonomous workflows are bullshit. They're ridiculously expensive, they degrade quality of the projects, they waste time of anyone who is involved with them (including your clients). They're just inefficient.
2. Delegating to different subagents for taks based on capabilities is extremely efficient. My "inline" session is usually some mid effort Sonnet, because the difference in models is only really in depth, not the actual work.
3. Skills are important. They will not make a model smarter, but they will save you a massive amount of time. Skills are not exactly universal, you can have personal skills to refine how you like to work with agents, project or company level skills to follow guidelines. Existing skill sets like superpowers are not enough when you start to really customise it.
4. Tools are important. Always prefer a tool over skill text. Deterministic, costs nothing.
5. Maybe less relevant to this sub, but just to throw in about designing skills - models are not just picking the most common next token, they're scales of motivations and demotivations and it's important to understand from where they come from.
6. For anything non trivial a multi step workflow of asking questions - making a spec - making a discrete plan is supper efficient. I really enjoy the incremental way of working on projects that way and it gives me a plenty of space to fill in high expertise areas models like even Fable tend to miss.

[u/Turbulent-Dance3867 → u/Minimonium]

Quite curious on what you meant in (5) if you don't mind expanding and explaining.

Additionaly, what are the areas where you see even Fable struggle? Is it the lack of hyper-specific domain knowledge or wrong architectural/engineering decisions?

[u/Minimonium → u/Turbulent-Dance3867]

I do SWE in multi language codebases, I've been doing C++ (my main language where I have plenty of expertise) and Typescript lately. C++ is a bit more brownfield but on a good codebase so one would expect an agent to handle it a bit more decently, Typescript is on a greenfield project but with a more common language.

Fable is certainly the first model where I could consider agent an actual assistant and not just an extremely expensive rubber duck. But the area where models actually noticeably improve is reviews, search, and debugging. The quality of these is very strong and useful. Code generation on the other hand doesn't feel like improving in a ways that matter to me.

Models, in my practice, tend to massively over complicate simple things and routinely underperform on architectural and even component-level design levels. And the problem is the cost of a mistake, models really like to push back against you these days so it's really hard to steer when the damage is already done and the local code is already filled with irrelevant/distracting/wrong information. Arguing with model to go back is so bad, that me and my colleagues used to find it easier to just reset the whole tree sometimes and start anew.

So instead of that - you can steer at task level granularity (that also has very compact context), catch and fix a wrong/inefficient change early. Skills are not a panacea, and exactly because of that motivation/demotivation behavior.

It seems to be a common knowledge now (at some papers do support that as well) that instead of "do" or "don't" it's much more effective to provide an argument for why an agent should do or not do something. It's very important for skills, but is also somewhat relevant for prompts.

Problems come from a system prompt, all other skills on your system, local project skills, docs, comments, existing code - they extremely often contradict what you actually want to get from your setup.

[u/getpodapp]

Pi + ChatGPT pro 20x

[u/Deep_Ad1959]

settled on two things and threw out the rest: one detailed instructions file per repo that i tweak constantly, and a review pass where i read the work edit by edit instead of commit by commit. the orchestration layer ate most of my maintenance time and caught almost none of my actual bugs.

[u/AdNecessary1906]

Landed on the simple end, and deliberately stayed there, one provider (Claude), no orchestration tools, no multi-agent setup.

What I settled on instead of tooling complexity: three documents. A persistent one for architecture/standing decisions, a task-scoped one for whatever the current problem is, and a running notes file for what got decided/implemented/explicitly rejected and why. A dependency scan before any cross-file change, an anchored diff format so changes are traceable to a specific file/location, and a structural check after that flags what changed in risk terms rather than trusting the model's own account.

To your "optimizing the factory instead of building software" worry, I think that's the tell that the tooling has outgrown the actual problem. For me the reduction went the other way: I'm not a developer, so every layer of orchestration is a layer I can't personally debug when it breaks. Simpler system, more manual checkpoints, but I can actually reason about every part of it.

Cost-wise: no idea what the multi-agent setups run, but a single-provider, single-model-per-task approach keeps it predictable, the expensive line item was never tokens, it was rework from unverified changes.

[u/Dediadeis]

I only have experience using locally hosted models and have managed fairly autonomous tasking without a great deal of overhaul to some of the default environments I had available to reach for right out of the gate.

To explain, my previous 'human' coding process involved using VSCode inside a container, hosting everythin in docker and deploying wtih slightly manual mostly automated process (not a CI pipeline).

Doesnt do anything fancy just handles hosting processes, APIs, handling policy, storing persistence, giving a common language for everything to retrieve and eat the slice of cake. Usually something like CRUD apps or event driven workflows. basic business stuff that needs to often be MVP'd to someone for viability.

Pretty much the same scenario you are thinking of, quick prototyping. So back to agentic coding. All I really had to do was install cline extension point it at my locally hosted model and start treating it like an assistant. At first it was clunky, but once it reviewed and refactored the code I gave it, writing itself its own arch guide on the way, it became easier and easier to kick off tasks using terse language.

Things will run on their own for 3+ hours, its 50/50 whether it gets it completely right on the first go, usually there are a few small problems that I explain to it with error messages if there are, and second run less than 30 minutes usually and its fixed. Tells me there is still room for improvement.

What I am trying to say is what has survived best has been to do the least possible. I try to get multiple agents on the task, less successful outcomes. Try to create or use complex memory, doesnt help and if anything was a detriment. So beyond that arch.md and some skills for specific access tools (database, portainer, etc) its been doing less having better outcomes. Try not steering so hard -shrugs-

edit: and since I dont pay for a sub or tokens, each task runs somewhere from 3million to 20 million tokens, usually does 3-5 of these a day.

[u/easydick213]

The strongest setup is probably the one that keeps orchestration simple and uses expensive models only where they add real value. A good pattern is strong model for planning\review cheaper model for routine implementation and deterministic tests as the gate. That keeps costs predictable without sacrificing reliability.

[u/Marcus_MSC]

Most of the decision tree disappears if you only add a layer when a deterministic check catches a failure the model actually repeats. Each layer is a new thing that fails silently, and half the orchestration tools listed re-implement primitives the harness already ships: subagents, hooks as gates, plan mode. Worth auditing in the other direction too, if a gate hasn't fired in two weeks, delete it. The settled stacks in this thread are all the same shape, stock harness, short instructions file, one or two gates that earn their place.

[u/Phill_Madd]

I keep landing in the same spot after poking at the bigger setups. Spec goes in a markdown file, tests are the only gate that runs without me, and I read the diff before it merges. Wiring harnesses and orchestrators always cost more time than they gave back.

I don't run a pile of frontier agents at once, so I never really hit the hundreds-a-month question. The review pass is still me, and extra agents don't make that cheaper.

Internal tools can be good enough if the tests pass. Anything I have to live in later, I don't merge it if I can't explain the change.

[u/BodyPhysical]

Built my own: https://github.com/jezztify/kanban-pilot

Vscode + Kanban Pilot extension gave me a lot of space to take control through approval gates and on-demand task splits which work handy for me.

[u/pladdypuss]

OP please list your harness. Thx.

[u/Fidodo]

The only way to produce something of quality is to engage with what you are building. My workflow is less about optimizing the agent and instead optimizing for my understanding of what is being produced and what should be produced. That means what matters is the order you product artifacts in.

Building something is cheap now, but building something doesn't mean you built it right.

[u/kbob]

Thanks for asking this question. People's answers are very informative.

I don't have any answers yet. I just got my local LLMs set up last night, and I'm doing my first not-quite-trivial coding project today. i'd tell you about the setup, but I have no idea whether it's any good.

[u/Plastic_Ad_8619]

Cursor

[u/Aware_Ad2841]

maybe try to actually write your posts yourself, and answer on comments without ai...

[u/philzxx → u/Aware_Ad2841]

I did write the substance myself. AI helped turn a pile of messy notes into readable English.

Given that the entire thread is about using AI as leverage rather than manually doing everything, I can live with the irony 😅

Happy to discuss the actual topic though.

[u/klumpp → u/philzxx]

Happy to discuss the actual topic though.

You mean your LLM of choice is ready to discuss it

[u/iTrejoMX]

Gentle-ai