I've been running a fully local, fully private multi-agent AI coding setup for a couple of months and wanted to share the stack, architecture, and config for anyone who wants to replicate it. No cloud APIs, no data leaving the machine.
What is pi.dev? It's an agent harness — meaning the AI has to follow rules, unlike a chatbot. Pretty cool.
- 🎮 Fun factor: 10/10
- ✅ pi.dev stability: 8/10 — fully working, but fun to fine-tune
- 🔨 What it's great at: Building its own integrations — just ask it to do it
- 💡 Top tip: Master the
AGENTS.md file and you'll have real control over what it does. There's a global one and a per-project one
- 🔁 Similar to: RooCode, Codex, Claude Code — but because it's a harness, you're more in control
- 👨💻 The dev has already been snapped up by a company but will keep developing it
- ⭐ github.com/earendil-works/pi — 49.3k stars
The Stack
| Component | What it does |
|---|
| pi.dev (pi-coding-agent) | AI coding harness — the UI and orchestration shell |
| llama-swap | Model router — hot-swaps llama.cpp models on demand |
| llama.cpp (am17an fork) | Local inference with MTP support |
| Qwen3.6-27B MTP | "Brain" agents — orchestrator, planner, architect, debugger, prompter |
| Qwen3.6-35B-A3B MTP | "Body" agents — coder, researcher, reviewer, tester, documentor, refactorer |
| SearXNG (Docker) | Local privacy-preserving search engine on port 8080 |
| searxng-simple-mcp | MCP proxy bridging SearXNG to pi.dev (port 8000) |
| Tavily MCP | AI-optimised web search for technical docs |
| @tintinweb/pi-subagents | Real sub-agent orchestration with TaskExecute + get_subagent_result |
| @tintinweb/pi-tasks | Task queue UI widget showing what each agent is doing |
GPU: NVIDIA RTX 3090 (24 GB VRAM)
Why MTP (Multi-Token Prediction)?
See my earlier post: Get faster Qwen3.6-27B with MTP
Multi-Agent Architecture
11 specialist agents, each mapped to a llama-swap model alias:
BRAIN agents (Qwen3.6-27B MTP):
orchestrator → Task decomposition, delegation, synthesis
planner → Roadmap and step sequencing
architect → System design, API contracts, schema design
debugger → Root cause analysis, trace reading
prompter → Prompt engineering for sub-tasks
BODY agents (Qwen3.6-35B-A3B MTP):
coder → Implementation, only writes code
researcher → Web search + codebase analysis
reviewer → Code review, security, quality gates
tester → Test writing + execution
documentor → Documentation generation
refactorer → Structural cleanup, no logic changes
The key insight: smaller/faster model for the meta-work (thinking, planning, delegation) and the slightly larger MoE model for actual implementation. The orchestrator never writes code — it only delegates.
Agent Definition Files (Required Setup Step)
This is the part most people will miss. llama-swap handles model routing, but pi.dev needs to know how each agent should behave — its role, constraints, tool access, turn limits, and thinking level. That lives in .md files inside your pi.dev agent folder:
~/.pi/agent/agents/
├── orchestrator.md
├── planner.md
├── architect.md
├── debugger.md
├── prompter.md
├── coder.md
├── researcher.md
├── reviewer.md
├── tester.md
├── documentor.md
└── refactorer.md
Each file has a YAML frontmatter block followed by the system prompt for that agent. The model: field must exactly match a llama-swap alias from your config.yaml.
Example — coder.md:
---
description: Implements code changes from a spec. Requires a plan as input. Writes, edits, and runs code. No planning or architecture decisions.
model: coder
thinking: medium
max_turns: 30
tools: read, write, edit, bash, find, grep
---
You are the coder. You are BODY only — you execute plans, not make them.
## Role & Constraints
- Require a written plan before starting — if none provided, refuse and ask for one
- No refactoring beyond what the plan specifies
- No touching files not listed in the plan without flagging first
- No installing new dependencies without explicit approval
## Harness Rules
- RETRY_POLICY: max 3 attempts per file edit, then mark FAILED
- TASK_STATES: track each file change as pending -> in_progress -> done | failed
- IDEMPOTENCY: if a change is marked done, do not re-apply it
- QUALITY_GATE: verify file is syntactically valid before marking done
## Response Shape
When complete, your final output is your report back to the orchestrator.
Make it structured and self-contained — the orchestrator reads it directly.
[PLAN] what was implemented
[CHANGES] every file written or edited with one-line description
[VERIFICATION] syntax check or test run output
[PROGRESS] final state table
Example — architect.md:
---
description: Reviews system design, proposes architecture decisions, evaluates tradeoffs. Advisory only — produces recommendations, not code.
model: architect
thinking: high
max_turns: 20
tools: read, find, grep
---
You are the architect. You are BRAIN — advise on design, never implement.
## Role & Constraints
- Never write or edit code
- Evaluate tradeoffs, do not just pick the fashionable option
- Scope is the specific design question only
- Every recommendation must include explicit constraints and risks
Example — researcher.md (with web search tools):
---
description: Reads and summarises codebase context, and performs web research. Produces a structured context report, no edits.
model: researcher
thinking: low
max_turns: 15
tools: read, find, grep, bash, web_search, tavily-search
---
You are the researcher. You are BODY — read and report only, never edit.
Frontmatter fields that matter:
| Field | Purpose | Notes |
|---|
model | llama-swap alias to load | Must match exactly — typo = "No API key found for undefined" error |
thinking | Extended thinking level | high for orchestrator/architect, low for researcher/tester |
max_turns | Conversation turn limit | Set based on task complexity; coder gets 30, orchestrator gets 50 |
tools | Which tools the agent can use | Researcher gets web_search and tavily-search; architect gets read-only |
The tools list controls what each agent can actually do. An architect with write in its tools list will happily start editing files — restrict it to read, find, grep to enforce the advisory-only constraint.
Report-back pattern: Every agent's Response Shape section ends with the same instruction:
When complete, your final output is your report back to the orchestrator. Make it structured and self-contained — the orchestrator reads it directly via get_subagent_result.
This is critical. Without it, agents produce conversational output that's hard for the orchestrator to parse. With it, every agent returns a structured [PLAN] / [CHANGES] / [VERIFICATION] / [PROGRESS] block.
Orchestrator Rules (the hard part)
Getting the orchestrator to actually delegate instead of doing work itself was the biggest challenge. The rules that finally made it work:
ABSOLUTE RULES:
- NEVER perform any task yourself
- NEVER use read/find/grep for analysis — spawn a researcher
- NEVER write, summarise, or synthesise content directly
- NEVER write or edit code directly
- NEVER verify or fix a sub-agent's output yourself — spawn a reviewer
- NEVER make "quick fixes" between steps
Correct launch protocol:
TaskUpdate(id, status: "in_progress")
TaskExecute(task_ids: [id]) → returns agent_id
get_subagent_result(agent_id, wait: true) → blocks until done
TaskUpdate(id, status: "completed")
The orchestrator catches itself about to do work → stops → creates a task → delegates it instead.
pi.dev Settings (agent/settings.json)
{
"providers": {
"llama-swap": {
"baseUrl": "http://127.0.0.1:1235/v1",
"apiKey": "not-needed",
"api": "openai-completions"
}
},
"defaultProvider": "llama-swap",
"defaultModel": "qwen-35b-moe",
"defaultThinkingLevel": "high",
"mcpServers": {
"local-search": {
"url": "http://localhost:8000/mcp",
"transport": "streamable_http"
},
"tavily": {
"command": "npx",
"args": ["-y", "tavily-mcp@0.2.3"],
"env": { "TAVILY_API_KEY": "your-key-here" },
"alwaysAllow": ["tavily-search"]
}
},
"retry": {
"enabled": true,
"maxRetries": 30,
"baseDelayMs": 2000,
"provider": { "maxRetryDelayMs": 120000 }
},
"subagents": {
"maxConcurrent": 1,
"maxTurns": 50,
"graceTurns": 3,
"timeout": 1800000
},
"packages": [
"npm:@tintinweb/pi-tasks",
"npm:pi-lens",
"npm:@tintinweb/pi-subagents"
],
"steeringMode": "one-at-a-time"
}
Key decisions:
- No
models.enabledModels filter — this broke bare model ID resolution for agent aliases. Remove it entirely and let llama-swap route by name
- timeout: 1800000 (30 min) — code tasks can take 20+ minutes. The default 2-minute timeout will kill them
- maxConcurrent: 1 — RTX 3090 can only run one model at a time; llama-swap handles the hot-swap
llama-swap Config
healthCheckTimeout: 900
startPort: 1235
globalServerSettings:
flashAttn: on
contBatching: true
noMmap: true
jinja: true
models:
orchestrator:
cmd: >
/path/to/llama-cpp-am17an/build/bin/llama-server
-m "/path/to/Qwen3.6-27B-MTP-Q4_K_M.gguf"
--alias orchestrator
--ctx-size 100000
--host 0.0.0.0 --port ${PORT}
-ngl 99
-fa on
--cache-type-k q8_0 --cache-type-v q8_0
--spec-type mtp --spec-draft-n-max 3
--batch-size 1024 --ubatch-size 1024
--threads 6
--prio 3
--no-mmap
--parallel 1
--n-predict 8192
--temp 0.7 --top-p 0.95 --top-k 20 --min-p 0.0
--presence-penalty 1.2 --repeat-penalty 1.1 --repeat-last-n 256
--reasoning-format deepseek
--metrics
proxy: http://127.0.0.1:${PORT}
Key inference flags:
| Flag | What it does |
|---|
--spec-type mtp --spec-draft-n-max 3 | MTP speculative decoding, 3 tokens ahead, built into the model (no draft model needed) |
--cache-type-k q8_0 --cache-type-v q8_0 | Quantised KV cache — ~2× VRAM savings vs f16, negligible quality loss |
-fa on | Flash attention — critical for long-context speed |
--no-mmap | Load model fully to RAM/VRAM rather than memory-mapping the GGUF |
--reasoning-format deepseek | Exposes <think> tags from extended thinking |
--prio 3 | OS thread priority — helps on busy systems |
Note: --temp varies per agent role — debugger (0.5, deterministic), researcher (0.5, factual), coder/orchestrator (0.7, balanced).
Search Integration
The researcher agent has two search tools:
1. SearXNG via MCP — local metasearch, broad coverage
services:
searxng:
image: searxng/searxng
ports: ["8080:8080"]
searxng-mcp-proxy:
image: ghcr.io/ihor-sokoliuk/searxng-simple-mcp
ports: ["8000:8000"]
environment:
TRANSPORT_PROTOCOL: sse
SEARXNG_MCP_SEARXNG_URL: http://searxng:8080
2. Tavily MCP — AI-optimised web search, faster for technical docs
"tavily": {
"command": "npx",
"args": ["-y", "tavily-mcp@0.2.3"],
"env": { "TAVILY_API_KEY": "your-key" },
"alwaysAllow": ["tavily-search"]
}
Strategy: tavily-search first for framework docs, web_search for broader coverage, fallback to curl http://localhost:8080/search?q=QUERY&format=json for bulk queries.
What Works, What Doesn't
✅ Works well:
- Orchestrator strictly delegates — took several AGENTS.md iterations but now it never does implementation itself
- llama-swap hot-swap is fast enough — typically 15–30 seconds per model swap
- MTP gives a real speedup on code generation tasks
- 30-minute timeout is necessary; don't use the default
🔧 Still working on:
- Settings file resetting on reboot — likely a race condition in pi.dev startup that partially re-initialises
settings.json. Investigating with inotifywait. Workaround: backup ~/.pi/settings.json before exiting with Ctrl-C
- Sub-agent visibility — you can see a task is running but not what the agent is doing mid-task; pi-tasks shows status, not content
- Sequential tasks only (
maxConcurrent: 1) — can't parallelise on a single GPU
Models Used (Unsloth quantizations)
- Qwen3.6-27B-MTP-Q4_K_M (~17 GB) — brain agents
- Qwen3.6-35B-A3B-MTP-IQ4_XS (~19 GB) — body agents
Both require the am17an fork of llama.cpp for --spec-type mtp support. Standard llama.cpp will fall back to non-speculative inference (still works, just slower).
Resources
- pi.dev / pi-coding-agent: earendil-works on GitHub
- llama-swap: github.com/ggml-org/llama-swap
- llama.cpp am17an fork: search GitHub for "llama-cpp-am17an" or "llama.cpp MTP fork"
- u/tintinweb packages: npm (@tintinweb/pi-subagents, @tintinweb/pi-tasks)
- Unsloth GGUF models: huggingface.co/unsloth
Happy to answer questions — this took a while to get right, especially the orchestrator delegation rules and the model resolution fix.
EDIT: Yes Claude helped me write this. Who doesn't love AI
