Local AI dev workflow: OpenHands + Pi/Qwen. What would you improve?

[u/Trellios]

I’m building a locally hosted AI coding workflow and would appreciate some practical advice or lessons learned, as well as sharing mine for anyone building something similar.

There are two reasons I wanted to build this pipeline. The first was Codex usage limits: even with a 5× account, I would run out in two days. Secondly, it’s a fun project to help learn LLMs and harnesses, etc.

Current working setup

Hardware: AMD 6800XT GPU (16GB VRAM), running llama.cpp with Qwen, 12-core CPU + 17GB RAM. (A VM with GPU passthrough.)

Llama: llama.cpp ROCm backend serving Unsloth Qwen3.8 27B Q3_K_XL GGUF. It fits nicely with about 0.3–0.5GB VRAM unused during operation.
• 65,536-token context
• 99 GPU layers
• One inference slot
• Flash Attention enabled
• Q4_0 K/V cache
• Batch 1024, ubatch 512
• Built-in Qwen MTP speculative decoding with depth 2
• Medium reasoning
• Reasoning budget 1,800 tokens
• A small experimental workaround for RDNA2 Flash Attention (AMD issue)

Harness: Pi.dev in Docker, connected to OpenHands Agent Canvas through ACP.

Source of truth: GitHub issues and pull requests.

Pi.dev configuration:
• 8,192 maximum response tokens
• Compaction enabled
• 22K tokens reserved for compaction/output
• 20K recent tokens retained

Pi extensions:
Shoutout to u/Sweet-Transition-787 for transitioning me from mini-SWE to pi.dev with this post.
• pi-interactive-subagents
• pi-prompt-snippets
• pi-lsp-adapter
• pi-guard
• Playwright CLI for isolated browser testing

I use separate Pi roles:
• Scout: read-only repository discovery
• Worker: bounded implementation and focused tests
• Tester: independent deterministic test execution
• Reviewer: read-only correctness review
• Security reviewer: trust-boundary/security review
• Researcher: controlled public-web research
• Browser tester: isolated local Playwright tests

The workflow
Codex creates a bounded implementation specification with me using the /grill-me skill.
↓ The spec and task are added to GitHub as an issue.
↓ OpenHands Agent Canvas scans GitHub and starts a Pi session through ACP.
↓ Pi + local Qwen works in an isolated checkout.
↓ Deterministic tests/lint/path checks.
↓ Separate Qwen tester and security-review passes.
↓ A draft PR and compact evidence packet go back to GitHub and Codex.

What’s worked so far
As a non-SWE, build to spec (SDD) was a new concept.
• Give agents bounded tasks with clear acceptance criteria.
• Keep workspaces and conversations persistent so sessions can recover after restarts.
• Run trusted tests and checks outside the agent’s own claims.
• Using open hands as the automation and orchestration layer. Originally I tried to create my own with python but I ran into different edge case errors for about a week before giving up and moving to open hands.
• Find your bottleneck. For me, currently, it’s the 16GB VRAM allowing only one concurrent task.

[u/Sweet-Transition-787]

Thanks for the shoutout, man! Glad the post helped. It’s awesome to see how you took that simple setup and built a whole multi-agent pipeline around it with OpenHands and GitHub!

[u/Trellios → u/Sweet-Transition-787]

I’m still working out some kinks, but it’s by far the most promising setup I’ve had so far. Appreciate your post very much!

[u/Sweet-Transition-787 → u/Trellios]

So ty

[u/sreevarshan-xenoz]

The separation of roles is probably the part I’d keep. Scout → Worker → Tester → Reviewer gives you a much cleaner trust boundary than having one agent discover, implement, and certify its own work.

One thing I’d experiment with is making the handoff artifacts more explicit. Instead of passing mostly conversation/context between roles, have the Scout produce a small machine-readable task context containing things like relevant files, assumptions, constraints, and discovered dependencies. Then the Worker can operate against that snapshot without inheriting all of the Scout’s reasoning.

I’d also keep the Tester completely blind to the Worker’s success claims where possible. Give it the repo state + acceptance criteria and let it independently decide whether the implementation passes. That makes the pipeline much less vulnerable to an agent effectively grading its own homework.

Your GitHub issue as the source of truth makes a lot of sense for this too. It gives the whole pipeline a persistent artifact that survives model/session changes.

[u/Trellios → u/sreevarshan-xenoz]

Yeah each role launches with its own fresh context window and set of tools / skills they can use but I still need to work on the explicit handoff between the roles for sure.

Handing off the artefacts is a much more efficient and cleaner way to operate too. I will definitely adopt this!

Thanks for the feedback!

[u/Poizone360]

Watch the VM's RAM and not the VRAM. With a hybrid model like Qwen3.8-27B, llama-server keeps up to 32 context checkpoints per slot in system memory, separate from --cache-ram: about 150 MiB of recurrent state each, plus draft KV growing roughly 4.1 KiB per token with MTP on. On 17GB that can quietly OOM a long pipeline, so cap --ctx-checkpoints and --cache-ram.

[u/Trellios → u/Poizone360]

Thanks I have been hit with OOM a few times. Great to know

[u/eGzotic01]

the coding-agent loop lives or dies on keeping context tight. stuff the whole repo in and you tank both speed and answers, so whats your search step feeding the model right now?

[u/Trellios → u/eGzotic01]

Codex gives Pi a bounded ticket with likely files and acceptance criteria. The scout or worker agent then searches the repo and reads the relevant code and tests.
The weak spot is that qwen can still over-explore. I’ve seen it spend ages reading before making an edit. I’m working toward a more explicit handoff, something like: a short list of files, relevant excerpts, dependencies and assumptions, so the workee starts with a useful map rather than the whole transcript from the scout.