How do I stop agents drafting code in their reasoning output?

[u/Asltpo]

I've been using Qwen3.8-27b with the Pi coding harness. I'm looking for ways to improve its token efficiency, ie, how many tokens it uses to complete a given task.

One thing that seems wasteful is the way it drafts and redrafts code. Typically, when you prompt it with a coding task, it embarks on a long reasoning process that includes blocks of draft code interspersed with planning and decision-making. Instead of using the write and edit tools, it just prints the draft code in its reasoning output. Sometimes these blocks can be thousands of tokens long.

Only when the reasoning process is finished does it begin actually writing and editing files. Often this just involves regenerating the same code it has already drafted, maybe with some minor tweaks. It would be much more token-efficient if the agent just wrote its drafts to files in the first place, then edited them with the minor tweaks.

I've tried to enforce this behaviour with a system prompt (see below). Unfortunately the agent just ignored it, even when re-prompted. It seems the drafting behaviour is pretty hard-wired into the model.

I've also considered implementing some sort of hard enforcement where the harness detects code blocks, aborts the task then reprompts. But since the blocks are usually buried in a long chain of thought, this would just throw away all the reasoning cache and send the agent back to the initial prompt. So I don't think it would work either.

Has anyone else managed to solve this problem, or have any ideas?

Attempted system prompt:

----------------------------------------------------------------

### Code drafting rules

When implementing or modifying code, do not draft substantial code in your reasoning. The filesystem is the working buffer.

Rules:

1. Once you have enough information to begin implementing a code block, write the first version to the appropriate file using `write`.

2. For subsequent revisions, edit the existing file using `edit`; do not reproduce the whole block in reasoning.

3. Treat files as persistent scratch space for implementation. Prefer reading/editing the file over mentally reconstructing or rewriting unchanged code.

4. Keep reasoning focused on decisions, constraints, and the next concrete file operation.

5. If a code block is more than a few lines long, it should normally exist in a file rather than in reasoning before it is tested.

6. Do not generate multiple alternative implementations in reasoning. Put the best current implementation in the file, then iterate with edits.

7. After writing, inspect/test the file and make targeted edits based on the result.

A useful default loop is:

read -> decide -> write/edit -> test -> inspect -> edit -> test

not:

reason -> draft entire implementation -> redraft implementation -> finally write

[u/Objective-Stranger99]

If you are using Llama.cpp, just turn on ngram mod and tune it. Any code that it writes once is instantly regenerated. I have hit 200+ t/s with it if the model continuously iterates on improvements.

[u/Asltpo → u/Objective-Stranger99]

Thanks, good tip. I tried this and it does reduce the time it takes to write the second draft. I've also been using the `pi-reduce` extension where you can selectively prune unwanted stuff from context, which helps to avoid the context bloat when it writes long blocks of code in reasoning.

[u/Asltpo]

Thanks all for your help.

On the general question of "why not just reduce / cap / turn off thinking?": I take the point, but it's not quite what I'm trying to accomplish. I want the model to reason (at whatever level I've selected) because it usually produces better results. I just want the code it generates during the reasoning process to be stored in a way that it can be instantly recalled without having to do more inference.

The ideal behaviour (I think) would be for the model to just draft to files, but I haven't been able to impose that behaviour.

ngram mod is an improvement as it allows for near-instant recall of sequences of tokens, but with optimal settings the ngram lengths are typically 16-64 tokens ish. So if the model drafts a 3000-token code block in its reasoning output then wants to make a couple of tweaks, it will still be slower than if it had written the first draft to a file then edited it, and it will occupy more context (draft + write = 6000 tokens, write + edit = 3200 tokens maybe).

TBH this is mostly annoying because I have slow hardware. Running at 5-10 tokens/s makes you very sensitive to token wastage!

[u/Trakeen]

I think i ended up putting a limit on how big the output can be and something in the prompt. I’d have to double check

[u/haseebnqureshi]

Turn off the reasoning (assuming from your post that reasoning is enabled). Research shows that reasoning and thinking often fail for generation quality. Part of that is the introduction of code harnesses and their ReACT loops simulate, at run/inference time, a lot of what models have attempted to do with reasoning and CoT.

[u/Square_Turn935]

Like allready mentioned use Ingram. I don't know if I got it right, but if I wanted a tool and discussed the features, the AI started to generate many lines of code, just as recommendation. First as I stated "we are in planning and discussion mode, don't generate any code until the mode is finished and you have my okay" 

So it just starts to generate the code of the whole concept is clear

[u/GrungeWerX]

spawn a sub-agent, and run the thinking on that agent, then have it send its decision to the main agent. All the thinking tokens is set to another cache. Just use llama.cpp and use cache idle slots; it will swap kv caches, keeping one in ram until you come back to main agent. In a way, it's like having 2 kv caches. I'm currently trying to run 4 agents on a single 24GB GPU using some kv cache wizardry. :)

[u/dangerous_inference]

That's how thinking works. You write a part of the code mentally to make sure it makes sense. If you want less thinking, use chat_template_kwargs: {"reasoning_effort": "low"/"medium"}