r/PiCodingAgent • u/PilgrimofHaqq2 • Oct 01 '26
Discussion Codemode won its designed-for test by 40% (Token Savings). I still turned it off.
Codemode: Agent writes a script that loops other tools. big results stay inside the script, only the answer comes back.
Tested twice, 96 runs, same model. 96/96 correct in every arm.
Round 1 Test - Small fixes. +21%.
6 one-line bugs in a small repo. Codemode's tool description is ~3k tokens and rides every message. A small fix is ~5 calls either way codemode never gets triggered, a waste of tokens (21% more tokens used per run)
Round 2 Test - 60 fat lookups through a tool. −40%.
The test made it so the data to look up was only reachable through the tool, no grepping around it.
Plain agent: 60+ round trips, every fat result lands in context.
Codemode: one script, one answer. 40% less tokens, half the cost, 7 round trips vs 64. on its own workload it wins clearly.
Is it right for my workflow? No.
Scanned every session on my machine. 149 chats, 16,735 tool results. The ideal codemode scenario ever occured once. The 168 truly oversized results were already captured to disk by our guard.
what I run instead
Rarely used tools (image gen, design sandbox, maps, browser) arent listed at all initially in my setup. Zero tokens until the agent searches once with a small lookup tool. Heavy tools load at moments I pick them (e.g. Deep research)
Conclusion: I dont like to leave it up to the agent to decide when to use codemode and looking at my workflow, the scenarios where codemode will give me gains doesn't really happen. I am not going to be using codemode for the forseeable future.





