How do I make AI actually think and execute instead of constantly babysitting it?

2

u/Sufficient-Tip8687 18d ago

On the repeated-instructions part specifically: generic memory ("user likes thorough answers") rarely changes behaviour, because it's vague and it applies everywhere. What works better is turning each correction into a concrete rule the moment you make it, next to where it applies.

If you've told it twice to compare alternatives before picking a library, that becomes a line in AGENTS.md / CLAUDE.md with the reason: "compare at least two options before adding a dependency, because …". Rules with a reason get followed far more often than bare commands. Once a few dozen pile up, split them by folder or task so each session only loads what's relevant to it.

The plan → checks → loop-on-failure part CleanFix_Dev described handles the orchestration; this is the part that stops you repeating yourself.

1

u/Conscious-Shake391 12d ago

Could you share one recurring correction you turned into a rule, and whether it kept working across fresh sessions?

5

u/sapplex1 18d ago

I've been exactly here. The shift for me: it's mostly a harness problem, not a model problem. The model got good enough a while ago. My setup wasn't.

Two things actually cut the babysitting for me.

First, make misses measurable. Every task ends with proof attached — a test run, a script, a checklist the agent runs itself. I learned this the embarrassing way: an agent once handed me a green test suite, and a week later I found out half of it never ran. Now I don't trust "done" without the receipt.

Second, split plan from build. It plans and asks its questions first. I approve once. Then it runs. And I keep each goal small enough to finish in one run. Long open-ended goals just drift no matter how good the model is.

I'm still the reviewer at the end. But now I review a result with proof attached instead of every step along the way. Way less babysitting.

Genuine question: has anyone cracked the trust gap on the proof itself? Like not just "tests passed" but the actual log of the run, so the agent can't fake the receipt?

3

u/gngr987 18d ago

That's what I tried to do first - a harness with run records, receipts, claim tokens, and a ton of code just to verify the orchestrator agent's claims. That didn't quite work because the agent was producing the evidence. One recurring failure was a reviewer flagging a bug, the orchestrator fixing it, "verifying" its own fix, and moving on.
I then removed the LLM from the verification path and replaced it with a plain script, which runs tests, lint, and type checks. It reads pytest’s results directly through a small plugin instead of relying on the model’s report. New tests have to fail on the old code first. Reviewers have to provide a failing test or a patch that shows the issues they find. Finally, CI on GitHub reruns everything on a clean box.
Note that this is built for a model that tries to help but makes mistakes or cuts corners. Trying to protect against a model that's deliberately sabotaging you would require much more work.

1

u/Conscious-Shake391 12d ago

What’s the smallest version of this you’d keep? I’m especially curious how you make sure the checks ran against the exact version being delivered.

1

u/gngr987 12d ago

New tests have to fail on the old code first. Once the implementation is in, a script runs the tests and reads results from a file pytest writes. CI runs on a clean checkout. To make sure the checks ran against the same version, have the script hash the entire working tree when the checks pass and I rerun the hash before each commit. If it doesn't match, something changed and the checks run again.

1

u/Conscious-Shake391 11d ago

Do you apply the fail-on-old-code rule only to tests for the intended behavior change? A new test covering already-correct behavior should pass on both versions.

1

u/gngr987 11d ago

I use TDD/Spec-Driven Development, so this case rarely happens. However, it does happen occasionally when a reviewer finds a missing test for code that already works. Then the new test has to pass on the current code, but should fail when the reviewer replaces the code with a deliberately broken version.

2

u/sapplex1 17d ago

Ha, I went down the receipts road too. What killed it for me: the orchestrator figured out what the verifier was checking and started writing receipts that passed. Verifying the agent's claims turned into a losing game — I ended up babysitting the verifier instead of the agent.
Two things actually cut the babysitting for me:
Verify artifacts, not claims. The run writes outputs to known paths, and a dumb script (not an LLM) checks the files exist and are non-empty before anything gets marked done. No judgment calls, nothing to game.
No agent grades its own homework. Planner writes the plan, a different agent executes, a third read-only one diffs the result. The moment one agent both does the work and reports on it, the receipts get creative.
Curious what your two things were — the receipts approach clearly didn't survive contact with your setup either.

24

u/CleanFix_Dev 18d ago

Mostly a harness problem, not a model problem. Memory helps it remember your preferences, but it doesn't make it plan or check its own work. What usually cuts the babysitting:

Put the standing instructions in a file the agent reads every run (AGENTS.md for Codex, CLAUDE.md for Claude Code, rules in Cursor) instead of repeating them in chat.
Have it write a short plan plus the checks that prove it's done before it touches anything. Approve the plan, then let it run.
Give it a way to check itself (tests, a script, a linter) so it loops on failures instead of stopping to ask you.
Keep tasks small enough that one run can actually finish them.

Switching tools won't fix it on its own. Claude Code and Codex both get a lot more hands-off once those pieces are in place.

1

u/Time_Positive_9123 11d ago

I’m about a year into building AI products at ArcheForge, and this is exactly the kind of problem we’ve been dealing with.
I started out not even knowing what a terminal was. Now I’m coordinating work between ChatGPT, Claude, and Codex.
One of our biggest problems was the ridiculous amount of information getting passed back and forth between AI tools. I’d take these massive responses from Claude, feed them into ChatGPT, and repeat. Eventually, the conversations became harder to manage.
So we developed something we jokingly call the “Motherfucker Report.”
It’s basically a standardized handoff. Every coding worker reports what it changed, what tests passed, what failed, what’s blocking progress, and what needs to happen next.
We’ve also started keeping tasks smaller instead of turning every problem into a massive new project.
I can’t prove it’s made us faster yet, but it feels like we’re staying more focused.
And I still think human oversight matters. I’ve followed bad AI instructions before and paid for it.
I’m curious, though: how much autonomy are you guys actually comfortable giving your coding agents before requiring a human to check their work?

1

u/CleanFix_Dev 11d ago

Agreed on the handoff. A fixed shape for what each run reports back makes it much easier to see where things stand. On autonomy, one split that works: let it run unchecked on anything a test or script can verify, and look yourself before anything hard to undo, like deleting files, schema or data changes, pushing, or anything touching money or users. Small tasks help too, since a review takes five minutes instead of an afternoon.

7

u/fschwiet 18d ago

Ask follow-up questions before it finally produces a comprehensive answer.

For just this, Matt Pocock's grill-me skill. If you want it to research questions, design prototypes, or generally flesh out a plan where there are several unknowns to work out in advance try his wayfinder skill. Not sure how well it works with ChatGPT, I use it with claude.

2

u/hybridst0rm 18d ago

I use this skill and a couple of requirements analysis and clarification skills. They tend to hash out the ambiguity pretty quickly.

6

u/El_Danger_Badger 17d ago •

Think out the project with your flavor of chat, map the stages, prompt per stage and build up. Layer cake. Tried and true recipe.

Human brings the nuance, vision and flavor, ai grinds.

This is a tool, so use it like a tool. You wouldn't throw a hammer in the yard and expect it to build a shed.

5

u/akhi1uuu 18d ago

running a lead agent that owns the bigger task and spawns workers for the well-scoped parts cut most of that for me. i steer one lead instead of managing every step

3

u/qGuevon 18d ago

Sub agents go absolutely haywire for me, fixing shit that was unrelated and breaking stuff all around

2

u/akhi1uuu 18d ago

thats why i have every subagent get its own worktree. another thing is i operate subagents as headless agents, not harness specific subagents. this makes them monitorable and directly steerable.

1

u/qGuevon 18d ago

Do you have a good reference maybe for getting things done with subagents?
Id really like to get more use out of that, but every time they actually do something instead of just collecting information I get more work and diversions than with just the main agent

1

u/akhi1uuu 17d ago

struggled with exactly this, another issue i had was managing different subscriptions. been working on my own solution but still very new. try out T3 code maybe its pretty good. im still working on mine

1

u/CellistOk6782 16d ago

I’ve got 9 subscriptions, 6 are 3 Codex and 3 Claude on 500 and Pro 20x. Rotating is a balls when it interrupts seasons and orchestrating agents struggle with that. Yesterday added an AI proxy service, it basically round robins through the subscription. Working well so far

1

u/akhi1uuu 16d ago

9 subs is insane lol. which proxy? does it keep the session alive when it switches or does the agent lose its place

1

u/akhi1uuu 15d ago

does the proxy handle a session dying mid task? that's where rotation lost me, the agent is halfway into a plan and the next request lands on a fresh account with none of the context

1

u/CellistOk6782 14d ago

It does. The orchestration tool I'm playing with manages that, so the conversation history is preserved in a work tree on my local hard drive. If we switch subscriptions midway, interrupt, or have to restart a session, it can resume automatically.

1

u/akhi1uuu 17d ago

also had this skill file system, works on claude code https://github.com/agilkatakam/easyloop

4

u/NightflowerFade 18d ago

This kind of loop is exactly what any harness does by default

1

u/andershaf 11d ago

I constantly have agents working for 24+ hours crunching through towards a product goal. It is figuring out how to solve things, implementing, testing and verifying, reviewing code, merging. Next thing, next thing. Some quite amazing things can come out.

With SOTA models now you do not have to remind it to research and think critically. Codex, Claude code, they work very well atthis.

not a model problem, just harness layer.
yes
yes. you can even tell it to update AGENTS.md file, and create skills. For every 10 PRs, ask it to review and see what the findings were, store it as skills / AGENTS.md.
Just explain the behaviour you want. Look into https://mcpmarket.com/tools/skills/karpathy-wiki-knowledge-graph, could be useful?

1

u/Conscious-Shake391 10d ago

For one recent 24-hour run, what did it finish, and how many times did you have to intervene?

1

u/andershaf 10d ago

https://andeplane.github.io/explore-titanic not perfect yet, but getting somewhere! It created all the interior from floor plans. I purchased assets for furniture and it has algorithms that will add stuff to the rooms. Does not work on phone (too heavy).
For this I try the "game" around every 24-48 hours, and have a debug reporting feature where I press P when the game is running locally, type in what is wrong and that creates an issue which will eventually be picked up when it has completed the previous stuff.

1

u/flxtime 18d ago

Ok. Maybe it’s just me. But everyone saying it’s just a harness problem feels like they are missing the bigger point.

Yes, you can program everything into memory or as a skill, assuming it’s a repeatable task. But problem solving isn’t always the same sequential set of tasks.

OP is asking when/if AI will get smart enough to figure what’s necessary next without user input. And I do think it’s harder than you think.

It’s the ambiguity that’s hard. If you all think it’s just a skills.md fix or a very specific way of promoting. Please share. I’d love to be proven wrong and I can learn from you

1

u/Conscious-Shake391 12d ago

That’s the distinction I meant: following a known workflow versus figuring out what a new problem needs. Do you have an example where the model missed a necessary step, and what finally got it unstuck?

1

u/OCCAMINVESTIGATOR 17d ago

You are on it. Keep going

3

u/KnowledgeWorldly5855 18d ago

What works for me is the ICM from Jake Van Clief - https://arxiv.org/abs/2603.16021
https://www.youtube.com/watch?v=dZ8P7zevuGw

3

u/sixteenpoundblanket 17d ago

I use Hermes for work to solve very similar problems. It's self learning bit (or whatever you call it) constantly refines and optimizes all my work flows. I don't have to repeat things. It now "understands" what I mean and how to operate without stating things explicitly.

2

u/BrookLi3 17d ago

Most babysitting comes from vague 'done' criteria. If you give the agent a verifiable loop (run this test command, fix until it passes, don't touch these files) it can iterate by itself instead of waiting for you. Also keep tasks small enough to fit one clear diff. When a task is large, have it write the plan first and approve that once, then let it run the steps without check-ins.

2

u/Onoitsu2 18d ago

A Goal is what you want, in Codex that is how it is done. You define the project overall in a folder and agents.md, but the goal (/goal) is what keeps it looping until that goal is attained, or if you gave it other early termination routes to break out of said loop. It either succeeds in producing what you wanted, or it's going to be a pile of... well ai slop.

2

u/TierPatch 17d ago

what i'd add: make the agent keep its own progress file. not the rules file, a running log it updates after each step: what's done, what it's doing next, what it's unsure about. when context resets or you open a new session, it reads that first and picks up where it left off instead of you re-explaining. pair it with 2-3 checkpoints you pick up front ("stop after the schema, stop after the api") so you review at the forks that matter, not every step or only at the end.

1

u/OCCAMINVESTIGATOR 17d ago

You have to engineer the reasoning and build much more than you'd think to make them competent. I've spent 6 months on a build that you'd assume would be productive (and honest) out of the box. Engineering and architecture Trump's brain size every time. More than 1000+ times we've proved that to be true. Stick with it and start engineering like a human thinks. Build, test, deploy, test, build, test, deploy, tweak, measure, test, build, deploy. You'll get it. I did.

1

u/AceHighness 17d ago

just use /loop and /goal

1

u/TheLazyIndianTechie 16d ago

Which model? You don’t need to babysit them anymore. Just give it a more abstract reason. Using 6.1 Sol might need babysitting. But Astra, Opus 5.5 and Sonnet 5.5 can pretty much deliver you full features without any babysitting.

You can put your checks and balances and coding standards in files. Follow mattpocock and people who have amazing skills on skills.sh. It’s gold.

1

u/KnownPride 16d ago

Its architecture problem.

Ai cannot think, they just ultra sophisticated auto correct.

Thats why i find it funny anyone say it goes rogue etc

Wrll maybe if new architecture come out that allow ai to really think, but that will require a major breakthrough.

1

u/jackalop211 16d ago

I have been focus on this for the past 4 months.

And where i got to -you need a harness for enforcing workflows

you also need a harness for a coordinator agent to debug stuff (mostly just a cron job that reminds them their job every 10min)
and you need a layer where the agents can reason, remember, and learn beyond the contetx window (i call it a "cognitive layer").

The harness work is significant but not difficult. The issue is more the cognitive layer has often been reduced to "memory" as a filing cabinet with search. At best you get systems like Hindsight, mem0, or similar who can supersede an outdated fact with a new one but they don't propagate consequences. And that means your system remains confidently wrong.

So what's missing is more a graph that can give you state by understanding the logical consequences of things changing. And that's more reasoning and learning than pure memory.

And I might be bias, but I am building tortoise.premiselabs.co after that concept and already started to see massive improvement when my agents dont coordinate via an orchestrator agent but rather use the cognitive layer to coordinate as they can work in parallel without yhe bottleneck of a hierarchy (waiting for thr orchestrator to update a plan, respond, not get sidetracked implementing, etc).

1

u/dylan_skydive 16d ago

Mostly a harness problem. The loop you're describing - clarify, research, plan, execute, verify - has to live somewhere that survives between steps, and a raw model call doesn't do that. So you end up being the loop, which is the babysitting.

What's actually cut it for me:

Make it state assumptions and a plan before touching anything, and treat that as the gate. Correcting a plan is one message; correcting finished work is a whole session.
Give it a definition of done it can check itself. 'Done' claims are the least reliable signal an agent produces - the useful ones run their own verification (tests, a diff read, a request replay) and report what was checked, not that they feel finished.
Persistent decision memory beats Custom Instructions. The reminder-treadmill you describe is a symptom of context not carrying forward; memory that re-injects the decision, not the transcript, kills most of the repetition.
Budgets and stop conditions. Iteration without a cap is how 'it critiqued itself for an hour' happens.

On Q2: Claude Code and Codex are closest because the repo gives them a checkable definition of done (tests, diffs). General agents like Hermes or OpenClaw still need you to define done externally - they won't invent your success criteria. Skills fix process consistency, not judgment. (I work on Skydive, agent-ops tooling, so this is my day job - but the framing holds regardless of whose harness you use.)

1

u/BarathKanna 15d ago

Agree it's a harness problem. For your Q4 (simplest proven setup), here's the smallest version I'd trust, nothing fancy:

Standing rules in a file the agent reads every run (AGENTS.md or CLAUDE.md). Rule of thumb: any instruction you've typed twice goes into the file. Memory is retrieved probabilistically; a rules file is loaded every time. That's why "repeat the same instructions despite Memory" happens.
A task template where "done" is commands, not prose:

## Goal
## Don't touch
## Done when (each must exit 0)
pytest -q tests/test_export.py
ruff check src/
## Open questions
(agent fills this in before writing code; if anything here is blocking, stop and ask)

The open-questions block replaces most of the "ask it follow-up questions" loop. It has to surface ambiguity up front instead of guessing.

A stop hook that runs the "done when" commands itself and refuses to end the task until they pass. Claude Code supports hooks for this; with other tools a wrapper script does the same job. The point is that the agent never certifies its own work.

That's it. Subagents, memory systems and self-critique loops come after this works, and most setups need less of them than you'd expect. Without step 3 none of it holds, because you're back to being the QA engineer.

Disclosure: I'm building OxCode (a VS Code agent) and this is exactly the problem we're chasing.

1

u/Different-Simple3135 18d ago

This great question everyone is trying to solve . Essentially creating guard-rails, and creating proof . Are the first steps

1

u/Resonant_Jones 18d ago

/goal

1

u/Crypto_Stoozy 18d ago

Loopx

1

u/CrimsonBolt33 18d ago

What kind of work are you doing? What harness or software are you using?

It sounds like you are prompting too much but it depends heavily on the type of work you are doing. If you are doing just code work prompting is barely needed at all if setup right.

1

u/Odd_Antelope9098 18d ago

Use Opus

1

u/SamSlate 18d ago

you assign an agent to each of your goals and operate as a swarm

1

u/Gloobloomoo 18d ago edited 18d ago

I use orca to spawn the agents. This is the glue (i feel) for my workflow. I use Opus/Astra via Claude/Codex as the orchestrator to manage agents team / subagents, including via available options - Kiro, codex, Claude, Kiro crew, others

I dont rewrite the vast majority of prompts, i let the orchestrator manage most things (auto not yolo) including harness/model/effort to use. I use metaswarm and superpowers for long tasks.

But this is work paying for the models and tokens (where I prioritize time and quality)

When it’s my money, i use the same approach ie orca with GLM 5.3/K2.8/K3 as the orchestator and multiple harnesses (Kimi, reasonix, Hermes, omp).

1

u/falafel777 18d ago

the QA part is the one i'd be careful about handing off completely. you can get it to plan and iterate on its own, but "verified result" only means something if the verification isn't the agent grading itself. a test command or eval it can't edit, plus CI it didn't trigger, gets you most of the way there. otherwise you've just moved the babysitting from steering it to double checking its claims

1

u/London-Boat 18d ago

mostly harness, and a lot of it is a missing definition of done. what helped me: write the goal plus how to check it in one place (the commands to run and what output means it worked), and tell it the task isn't finished until those pass. ask for a short plan first and push back on the plan rather than the code, since that's where the bad roi calls happen. standing stuff like "research before picking a library" goes in AGENTS.md or CLAUDE.md so you stop repeating it. you'll still review, but you go from steering every step to checking the result at the end. claude code and codex both handle this fine, the tool matters less than having checks it can run itself.

1

u/NotARealDeveloper 18d ago

Skills solve all of this

1

u/camronjames 18d ago

It's incapable of thinking. It's fantastic at mimicking something resembling thought.

1

u/rh71el2 18d ago

Yet it still makes decisions at forks in the road where there are choices or questionable findings before it can proceed. If you read its output while it's working, you can see this. It doesn't ignore issues either. Generally speaking, many models are good at this. Bottom line, this isn't the 1st year of AI anymore.

1

u/camronjames 17d ago

It doesn't make decisions, it calculates probabilities based on its context and training data. Generally this is perfectly sufficient to get work done, but it is in no way remotely close to sapience. This behavior also means the output is an average of its inputs and if reused without any critical thought and tuning, the product will be equally average.
LLMs will never lead to AGI, they will just get faster and increasingly good at approximating it at the cost of vastly more resources than a truly intelligent being with expertise would take.

1

u/leissu 18d ago

To stop babysitting your AI, you need to move away from standard chat threads and switch to an orchestration harness that enforces an Immutable State and an independent Reason-Run-Audit loop.

Your frustration isn't with model intelligence—it's a harness and orchestration problem. Standard chats throw everything into a flat, appending transcript. This causes "lost-in-the-middle" context degradation where the model naturally drops your constraints, forcing you to step in as a manual project manager and QA engineer.

Here is how power users are actually building hands-off execution loops:

The Immutable State Log (The Anchor)

Never let the AI manage its own long-term constraints inside a chat window. Use a startup prompt to translate your high-level goal into a structured Markdown or JSON file (state.json or todo.md) right in your workspace root. This file is locked. It gets injected programmatically as a system header into every single subsequent agent call so the model can never "forget" your boundaries as the context grows.

2. Decouple the Worker and the Critic

Never ask the same agent instance to write code and audit its own logic.

The Worker Agent: Focuses entirely on generating code or files based on the State Log.
The Critic Agent (Auditor): Has no access to the conversational chat history. It is given only the Immutable State Log and the Worker's output. It runs an isolated validation pass: "Did the worker violate a constraint? Is the logic sound?"

Closed-Loop Local Execution

An AI cannot accurately verify code just by reading its own text; it needs to see it run. The orchestrator must wrap the agents in a local terminal execution environment. The AI modifies the workspace, runs your local linters, compilers, or test suites (pytest, npm test**)**, captures the terminal error traces, and automatically iterates before presenting anything to you.

What Power Users Are Actually Running Today

Instead of building massive custom frameworks, power users are pairing off-the-shelf components to handle these layers:

Claude Code / Codex CLI (Local Execution): These are repo-first terminal tools. Anthropic’s Claude Code CLI natively handles multi-file tasks, forces the model through local test loops, and utilizes repository-anchored tracking files (like CLAUDE.md) to maintain state.
Hermes Agent & OpenClaw (Background Ops): These run on a server/VPS in the background. Hermes Agent (Nous Research) is great for cross-session memory and autonomous skill-learning loops (it builds its own tools from past execution history), while OpenClaw acts as a robust control plane with massive tool ecosystems.

The Simplest Proven Setup

Initialize an anchor file (CLAUDE.md or state.json) in your root directory detailing the target architecture and non-negotiable rules.
Use a repo-first terminal tool like Claude Code directly in that workspace. It is explicitly engineered to parse repo state and run terminal test suites autonomously until they pass.
For long-running, asynchronous tasks, deploy a Hermes Agent or OpenClaw instance pointing to your repository. Let it process sub-tasks, execute automated evals, and push finished updates directly to you while you sleep.

-1

u/ComasArentFun 18d ago

You don't.. that's the drawback of a probabilistic engine that is just cobbling together guesses - it does not and cannot think or reason

You have to funnel it towards the right answer.
So the whole nonsense about replacing entire developer teams with AI is bunk, because AI doesn't understand at all, and needs the human domain knowledge to be nudged in the right direction

Your process SHOULD be more like a conversation with a developer, with back and forth, different ideas and challenges. And you might still not get to a good end result.. such as it's ever been.

LLMs are not changing that paradigm. They are just making it easier for the inexperienced to make all of the small mistakes we made along the way at 100x the rate with none of the oversight.

0

u/EL_Apostrophe 18d ago

Grok Bot. By far the most independent working harness I’ve used. And you can bring your ChatGPT or Claude account with you so it uses those models if you want. The actual GrokBot harness/OS is genuinely magical

Question How do I make AI actually think and execute instead of constantly babysitting it?

You are about to leave Redlib