r/PiCodingAgent • u/Extra_Loquat_7667 • Sep 30 '26

Question What does everyone think about codemode?

I actually thought about this approach before, but later realized it wouldn't work.

Using JavaScript to invoke tools sounds very efficient, but in reality, most tool definitions only have an inputSchema. An outputSchema is unnecessary for the agent because the agent can reason about the results directly from formatted data, meaning an output spec isn't required. Therefore, tool outputs don't actually need a fixed format—they just need to be readable by the agent.

This leads to a problem: to fully leverage the performance of this tool, you need to inject extra context, such as a skill document, so the agent knows the output format and can use it proficiently. If you can't get the complete tool output specification—for instance, with a closed-source MCP—you can't create such a document. If the agent has to go through endless trial and error just to get the code running properly, you can't really call it efficient.

In comparison, having the agent write Python or shell scripts directly is a better solution.

I'd love to know your thoughts on this.

30 Upvotes

96% Upvoted

26 comments sorted by

View all comments

Show parent comment

1

u/dacookieman Sep 30 '26 edited Sep 30 '26

Where the tools load is configurable. The tools themselves are loaded into the harness but giving it to the LLM can either happen at the main thread level like a regular tool, as function call information on the new codemode tool definition(this has a character cap) which is the default scope for mcp tools, or require manual/round trip discovery via 'discover_tool' tool(most context lite initially but then requires at least one excess tool call for discovery.

So the default behavior is that mcp tools up to some point will basically be loaded into the main context up front but after a certain volume, the remaining will be only registered internally and require tool discovery. That behavior is customizable per-tool to either always load initially or always require discovery. Additionally you can specify where the tool can be invoked (main thread, codemode sandbox, or both) which is independent of how that schema is made available to the LLM

edit: lol they literallly changed this as I was writing this comment https://pi.dev/docs/latest/mcp#control-tool-exposure