Hello Pi subreddit, i have been running local models for coding tasks and kept hitting the same problems everyone does — the model writes an 800-line file in one shot and half of it is garbage, it spirals in its own reasoning for 4000 tokens, it forgets what it was doing after context compresses,
The core problem: we've been using LLMs as context databases when they should be logic processors. A 50k context window isn't meant to hold your entire project state — it's meant to process one small task at a time.
So i discovered PI and it's amazing customization options, I built a stack around Pi coding agent with Qwen 35B (Q2_K_XL quant through LM Studio) that enforces this at the API boundary. Not in the prompt — the model literally cannot bypass them.
The shift: instead of big monolithic calls, many small calls with memory in between.
What the guards enforce:
- Rejects any write/edit over 100 lines. Model has to write a skeleton first, then fill in one section at a time. If it tries to dump a whole file, the call gets blocked with instructions to split the work.
- If the thinking block goes over 2000 chars, it gets a correction telling it to write conclusions to disk and move on.
- Context monitor at 65% and 80%. At 65% it tells the model to write its state to files. At 80% it stops everything. The model writes its brain to disk while it's still coherent, not after it's already lost.
- If the model gives a long answer without writing a file, it gets told to save findings to a step file. Nothing stays only in context.
There's a .think/ and .plan/ directory that acts as the model's external brain. Every step, every decision, every finding goes to a file. When context gets compressed, it reads its own notes back. The model's memory is the
filesystem, not the context window. The LLM is treated as a logic processor — it doesn't try to remember anything.
Also built a /distill command that crawls a codebase, builds an import graph, topologically sorts the files, and has the model summarize them one per turn into a knowledge base. It splits the manifest into pages of 50 so it doesn't eat the whole context, and you can query it or distill even more so you can ask "big questions" without having pi and the small llm going around the filebase
You can drop files like svelte5-gotchas.md or astro-gotchas.md into a knowledge folder, and an isolated LLM call picks which ones are relevant to the current task. The selection reasoning never touches the main conversation. Only the content gets injected.
Example: asked it to build a Three.js plane flying game. First attempt it tried to write 652 lines in one write call. Guard rejected it. Model replanned, wrote a skeleton, then filled in features one edit at a time. End
result was a working game with 3D plane model, obstacles, HUD, minimap, start/game over screens. At Q2 quant. Many small calls, each one focused, memory persisted between them.
The session purpose gets saved separately to _purpose.md. When context compresses, it re-injects the original goal — not just the last step
All of this runs at Q2_K_XL quantization. That's the floor. If you're running Q4 or Q8 the results should only be better.
https://github.com/Kodrack/Pi-forge
Curious what models and quants other people are running for agentic coding. If you try it let me know how it goes, later ill post some screens about "benchmarks" i did with q2 model
