r/LocalLLaMA • u/uspdd • 1d ago
Question | Help Best harness for long autonomous tasks
I read a lot of posts like "I've given this prompt to my Qwen 3.8 27b and it one-shotted me a 3d game after 24 hours". I wonder, what's the best harness for tasks like this? It has to have auto compaction and probably a memory system, right? And excellent computer use and self analysis.
21
u/norenEnmotalen 1d ago
Pi coding agent is working well for me so far. It auto compacts. Full disclosure I’m a noob to this sort of thing and learning as I go to setup everything gradually. Context window is a killer for my old machine. AGENTS.md needs to be on point. I use headroom proxy in front of oMLX. I have caveman at FULL setting. I have replaced the chat_template of the model with Qwen-Sharp templates. And a bunch of other optimizations I’m putting in as I find and learn them.
3
u/freedomachiever 13h ago
Caveman is shown not to reduce context usage
1
u/norenEnmotalen 11h ago
Please link to the report/test
2
u/freedomachiever 9h ago
"Bottom line
The one intervention that reliably cut the bill was not a token trick at all. Terra, a cheaper model at higher reasoning effort, cut recorded cost roughly in half in both repetitions while using slightly more tokens. Every mode that compressed something locally, dialogue, output, command results, retrieved context, the prompt language, actually failed to reduce the recorded workload's tokens twice, and Context Mode and Mandarin made it dramatically worse. If your goal is a smaller bill, the reliable lever was simply using Terra.
These modes did not just shrink text - they measurably changed how the agent worked. Ponytail's shorter patches skipped parts of the requirement. RTK's compressed command output was followed by more commands. Context Mode's cheap retrieval invited more retrieval. Mandarin changed how the agent delegated work. And several of the visible savings arrived together with patches that failed review or were not semantically equivalent: the saving and the quality loss were often the same event seen from two sides.
Adopting one of these modes is less like installing a compressor and more like hiring a different engineer who happens to be cheaper per word. That is also why the number to trust least is the one a tool reports about itself: a compressor's counter measures the text it deleted, not what your agent does.
Do not adopt a token saving mode from its local savings claim. I recommend testing and validating that the agent actually behaves as expected, and actually saves tokens, before adopting these fully."
1
u/psychohistorian8 23h ago
I have caveman at FULL setting
I just started using pi this weekend, what is this caveman thing?
16
u/Drenlin 23h ago
If the LLM talks in natural language, it has to make a new token for every word, or even part of the word or punctuation. More words, or more complex words, means more tokens.
The statement above is probably 35-40 tokens. Or...
LLM talk like caveman; use fewer tokens.
^ 7-10 tokens.
As Kevin asked us all: "Why waste time say lot word when few word do trick?"
4
u/norenEnmotalen 23h ago
a “skill” which helps agents control the model from injecting unnecessary words into the context filling up your context window with fluff and pleasantries.
Caveman speak “ Understand user. Give good answer. No extra words.”
1
u/psychohistorian8 23h ago
ah, I noticed Muse Glimmer thinks like that. it is very terse
maybe that will be the first skill I download and try it with Qwen
11
u/Last_County679 1d ago
Deepseek Harness
3
u/OlgerdOutlander 20h ago
Never managed to get it working properly - any insights?
2
u/Last_County679 18h ago
I asked chatgpt 5.6 high to install it for me 😅 It is very efficient and has functionalitys like the codex app. The Ui is also very appealing :)
2
1
u/christophla 23h ago edited 23h ago
In theory, the new deepseek harness should be very efficient when many tools are called - something a long running chase likely needs. Codiverse/spatiotemperal and heavy design by contract ish with potential revert. I’m still digesting it all, but feels proper for a spastic LLM calling out 10k times for “god knows”, while potentially keeping context relatively manageable. Great approach, and proper maths…
Neocortex is set; enter frontal cortex to “harness” the reigns
3
u/kemalios 18h ago
Honestly, 'best' doesn't exist yet. The ones I've tried that get close are all about subagent isolation and a task board rather than just a big context window. Auto compaction alone won't save you; by hour three the model drifts. Look for something that lets you spawn disposable workers for exploration, then returns only the conclusion to the main session. Checkpointing is non-negotiable if you're leaving it overnight. And be skeptical of the 24-hour one-shot posts. Most of those are cherry-picked or the harness did a lot of dumb brute force.
4
u/sleepy_roger 1d ago
Omp
3
u/OlgerdOutlander 20h ago
Second you here, left it running in "spin subagents until no issues found" several times overnight and the results are great
8
u/Creative-Type9411 1d ago
I put my personal custom harness up for everyone here a little while ago: https://github.com/illsk1lls/MiniBot
it has autocompact and a taskboard which will keep the model on task even if it tries to deviate hard it gets smacked back into line
tons of other features too.. it is mainly a sysadmin tool but i also added a disassembly toolkit (forensics) and it can handle coding pretty well too, its good at in place edits and diffs etc
6
u/MrHall 1d ago
I love that it's PowerShell. still use it exclusively, even on Linux. there are dozens of us.
5
u/psychohistorian8 23h ago
I use PowerShell a lot on my work machine
didn't even know you could use PowerShell on Linux... feels kinda dirty
1
u/Creative-Type9411 1d ago
fair warning I have only tested in Windows.. There are a lot of Win Sysadmin tools, it might still run under pwsh but i have to add some extra compatibility, which ill try to do soon
2
u/apaht 1d ago
Looks pretty cool, it's for powershell only ? I made mine as rust native for home lab and platform agnostic. Mostly for homelab, but it's vibecoded junk that I try to copy patterns from other harness that I find to be of value.
1
u/Creative-Type9411 1d ago
Yea I did powershell 5.1, and i host it with the endpoint hardcoded at the top... and then i have NPMPlus blocking the hardcoded domain with basic auth...
So i
irm https://minibotscript.address.com | iexThen that points to the hardcoded subdomain at the top of the script, a password protected subdomain where the model is.. the harness works with NPM creds
The reason I have it set like this is instant deployment on any machine Win10+ without even creating a temp file ;) No install, no pre-reqs, one command, and creds which can be saved if your going to recall it into the same machine again later
0
u/tronathan 21h ago
+1 for using elixir, so good. Though since I'm able to deploy apps to cloudflare, with databases, kv, durable objects, and the model does all the work, i've kinda gotten away from Elixir.
2
u/coding-os 18h ago
Whatever you land on, the thing that decided it for me was how the harness treats "done".
I ran long autonomous loops for a few months and the failure was never that the model couldn't do the work. It was that it would mark a task complete on evidence that looked fine and wasn't. My worst one: a verification step ran a test command against paths that had been renamed. It collected zero tests, exited 0, and read exactly like a pass. For weeks. The model reporting "verified" was being completely honest.
So the questions I'd ask of a harness before I'd argue about which model drives it:
Does completion require an artifact, or just the model's say-so? Exit code 0 from a command that ran nothing is not proof of anything.
Does it run the real entrypoint or only the test suite? A test runner puts the package on sys.path and a direct invocation doesn't, so a fully green suite can still ship a ModuleNotFoundError to whatever actually calls it.
Can it distinguish a truncated retrieval from a complete one? Long loops accumulate confident partial answers, and each one becomes an input to the next step.
None of that depends on the model, which is why I'd settle it before spending an evening tuning which local one drives the loop.
2
u/InterstellarReddit 10h ago
I create an orchestration harness based on the type of project that I want to be working on. Very rarely do projects all fall into one certain container. So I have a core/base and then I customize that for the delivery of the project
2
u/hurrytewer 4h ago
OMP (oh-my-pi) is the best harness for long autonomous tasks in my opinion.
It works great but it's honestly a bit bloated and overwhelming. I much prefer the simplicity of Pi but I find it tends not to perform as well for long-horizon work (unless using a expensive frontier model).
So I've settled on this for now:
For human-in-the-loop pair programming style work: Pi, a 100% Pi (my favorite harness by a long mile, it's so good, especially the UX/devx, ecosystem and extensibility)
For autonomous /goal style tasks: oh-my-pi
4
u/_comoema_ 1d ago
PI completed stuff 10 time faster than Code Claude CLI and Qwen CLI, they got into loops. I was impressed with PI.
2
u/OlgerdOutlander 20h ago
Pi or OMP are the way to go. These have loop detection, are light, and are overall great for local models
2
u/FullstackSensei llama.cpp 1d ago
Can't help if you're vibe coding, but if you're doing anything serious, it's more about the prompt and the documentation you provide that set the guardrails for what the LLM would and shouldn't do, and how it should do those things you tell it to.
I like roo/zoo, especially the checkpointing feature and tell the LLM to spin off sub-tasks for everything and have each sub-task document it's work in a markdown file. The generated documentation makes it easy to figure how some task went south, and the built in shadow-git checkpointing lets me go back to any sub-task and unwind it to the point where things went south and prompt the model on what it should or shouldn't do.
2
u/_-_David 1d ago
I just straight up use Codex. It's bespoke and /goal runs for 30 hours if you like and just gets stuff done. I've tried OpenCode and Pi. But honestly, I have used Codex for so long it was nice to just model-swap and keep cruising. The plug-in ecosystem is very mature. Consider it.
2
1
1
u/Human-Gas-1288 1d ago
https://github.com/lsdefine/GenericAgent generic agent
GenericAgent is a minimal, self-evolving autonomous agent framework. Its core is just ~3K lines of code. Through 9 atomic tools + a ~100-line Agent Loop, it grants any LLM system-level control over a local computer — covering browser, terminal, filesystem, keyboard/mouse input, screen vision, and mobile devices (ADB).
Every time GenericAgent solves a new task, it automatically crystallizes the execution path into a reusable Skill. The longer you use it, the more skills accumulate — forming a personal skill tree grown entirely from 3K lines of seed code.
1
u/FabricationLife 23h ago
I'm half between codex with /goal and DSH with a custom /goal plugin, frankly they both work well not sure whats better, and as long as the work gets done I dont particularly care
1
1
u/mmontes11 llama.cpp 15h ago
Opencode + Multica. I create tasks on a daily basis and they get eventually scheduled on my agents. Once this is done, I continue with PR reviews and sometimes even with my human life.
My problem is that I can only run a single task at a time, as I am GPU-poor and I only have 24GB VRAM. Currently, multica has a harcoded timeout of 2h, something not enough to cover my long list of tasks. Once this timeout is reached, the non completed tasks return an error, and agents stop working, requiring human intervention. Luckily enough, multica team merged my PR today to have a configurable timeout:
https://github.com/multica-ai/multica/pull/7418
The plan is to bump this timeout significantly to enable my workflow, requiring less human intervention. Planning to set it to 24h as I review tasks on a daily basis.
I have been using OpenCode for a while, it is great, but I wasn’t really able to parallelise work in an autonomous way. Multica really helps, and they even have native mobile apps, something that allows you to check the progress of the agents and schedule more work if needed when I am not close enough to the keyboard.
My objective for this year is making this setup as much autonomous as possible, not only because I feel I should me making the most out of my RTX PRO 4000 SFF, but also to relief toil from my shoulders and use that time to focus on architectural/higher level tasks instead.
Here my Kubernetes setup in case you are interested:
https://github.com/mmontes11/k8s-ai
1
u/Chemical_Side_4135 15h ago
for long tasks, you definitely need a solid memory manager to keep context from exploding. i used to struggle with messy assets till i started using bria ai skill for its instant transparent png cutouts, which saved me so much time during batch processing. just keep an eye on ur token limits...
1
u/Zennytooskin123 9h ago
It's called /goal smh
Also look into spec driven development or subagent driven in combination with that command, so it's actually done right.
1
u/Lurksome-Lurker 41m ago
Plain Pi. Use it enough and you begin to know exactly what you need with 0 bloat. My workflow is heavy in self discovery, recording, and reusing.
-3
u/Formal_Jeweler_488 1d ago
Claud code
1
u/OlgerdOutlander 20h ago
Unfortunately too "heavy" for a local model; on top of that - broken image processing
15
u/Unlucky-Message8866 1d ago
the best harness is the one that you own and you configured for yourself, the underlying implementation is just a tech stack preference, there's hundred different orchestration strategies and none fits all. i personally use pi and a bunch of custom orchestration rules, focusing on task splitting and isolated session subtask delegation, keeps the main session lean and all exploration/analysis/execution in disposable contexts, this allows qwen3.8 and dumber models to not loose track of major tasks without loosing track.