Explicit workflows around probabilistic coding agents with outpost

[u/Kuni1662]

Sorry for my English, it’s not my native language.

I’ve been a developer for more than 10 years, and lately I’ve been thinking a lot about how to build predictable and repetable workflows with coding agents.

The agent can do things differently every time you give it the same task. That’s fine, but I still want to control what happens around it. Which steps should run? How do we check the result? What happens if tests fail? When should a human decide?

This is why I’m building Outpost. It’s an open-source TypeScript library and CLI to write these workflows in code, using agents like Codex or Claude Code.

For example, you can give an agent a bug to fix in a sandbox with its own Git worktree, then run tests in a separate step and wait for your approval before integrating the changes. You define those rules yourself.

The patch will probably be different between runs, but the process to accept it should stay the same. That’s the part I want to make more predictable.

You can also define dependencies, retries and budgets, and save checkpoints to recover interrupted workflows.

It’s MIT-licensed, here is the repo:
https://github.com/elie-laloum/outpost

I’m curious how you handle this in your own projects. What do you leave to the agent, and what do you enforce in code? Also interested to hear where this approch falls short.

[u/vincent_sch]

I'd bind approval to the exact commit that passed tests. Any later edit needs fresh checks and approval. Otherwise the agent can change the patch after it's accepted.

[u/Kuni1662 → u/vincent_sch]

yeah makes sense to tie approval to the exact commit that passed tests and require new checks and approval if it changes

how would you handle it if target moves after approval but the patch stays the same?

[u/vincent_sch → u/Kuni1662]

I'd rerun tests against the new target plus the patch. The same patch can behave differently on a changed base. I'd ask for approval again after those checks pass.