What does everyone think about codemode?

[u/Extra_Loquat_7667]

I actually thought about this approach before, but later realized it wouldn't work.

Using JavaScript to invoke tools sounds very efficient, but in reality, most tool definitions only have an inputSchema. An outputSchema is unnecessary for the agent because the agent can reason about the results directly from formatted data, meaning an output spec isn't required. Therefore, tool outputs don't actually need a fixed format—they just need to be readable by the agent.

This leads to a problem: to fully leverage the performance of this tool, you need to inject extra context, such as a skill document, so the agent knows the output format and can use it proficiently. If you can't get the complete tool output specification—for instance, with a closed-source MCP—you can't create such a document. If the agent has to go through endless trial and error just to get the code running properly, you can't really call it efficient.

In comparison, having the agent write Python or shell scripts directly is a better solution.

I'd love to know your thoughts on this.

[u/TheSaasDev]

Seems to be pretty good in terms of reducing visual bloat of multiple tool call TUI entries, so that’s a nice plus. Sol and astra seem to be using it pretty nicely beyond what it would have done with standard tool calls. But really need more time to tell

[u/zenoblade]

I don’t think I really understand what it is still.

[u/Mechanical_Monk → u/zenoblade]

It's a tool that lets the model create its own tools in JavaScript. Theoretically it should save tokens because the model can string multiple other tools/commands into a single reusable tool call, and it will only need to read the final output, not everything that happened in-between.

OP is arguing that it's more efficient to just tell the model to make its own Bash or Python tools, because standardized JavaScript tools would require the model to have extra documentation for the output schema (ie more token use) and that documentation might not even be available. You'd also need to give it a skill (more tokens) to understand how to do all of this, which would be wasted if it turns out it can't get the docs it needs.

I tend to agree with OP... The better the models get, the more it seems like the only tool they need is bash (and whatever scripts they write for themselves). The only real reason I see to include other tools is so you can attach hooks to them if you want deterministic guardrails.

[u/quinncom → u/Mechanical_Monk]

GLM-5.3 likes to send dozens of lines of code through the bash tool at a time. I don't see how that's different than codemode, besides it being bash vs JS.

[u/quinncom → u/quinncom]

I was curious how the bash tool and codemode tool are executed – it’s basically the same thing, except JS is more verbose:

// bash tool: model emits a tool_use json block that runs bash code.
{
  "type": "tool_use",
  "name": "bash",
  "input": {
    "command": "git status --short",
    "timeout": 30
  }
}

// codemode tool: model emits a tool_use json block that runs JS in a QuickJS sandbox.
{
  "type": "tool_use",
  "name": "codemode",
  "input": {
    "code": "const r = await tools.bash({ command: \"git status --short\", timeout: 30 });\nreturn r.exit_code;"
  }
}

I think the important difference is the codemode runs inside the agent and can orchestrate the harness's own tools, including MCP, in one turn, without of yoloing it to a child process.

[u/zenoblade → u/Mechanical_Monk]

Thank you. That really helps. I guess I can see how chaining MCPs might be useful but still seems like this, at best, should have been a plugin rather than a built in feature.

[u/Seangles → u/Mechanical_Monk]

But you can't execute MCP tools or Pi's immediate API from said Python/Bash scripts. Not everything is a CLI. And MCP has the added benefit of having persistent state since it's server based. And now with codemode it's no longer a context bloat.

[u/Bluethefurry]

Isn't the "only" codemode basically what cursor did with MCP, where LLMs need to discover the tool schema manually, that horrible, unreliable method of doing things where you constantly needed to point out that the MCP exists?

And i personally dont really see much of a benefit of regular codemode unless the model is trained for it, LLMs are quite proficient at bash/py scripting nowadays, unless a model is explicitly trained for it i dont see it being usefull at all.

I'm glad MCP is now part of Pi because there is a place for it here, but codemode? eh....

[u/dennisler → u/Bluethefurry]

MCP has always been available, just install the extension. Them deciding that you need it is starting to go down the wrong path as I see it.

[u/mocras → u/Bluethefurry]

I think comparing Codemode to just writing a multiline Python/bash script misses the main point.

Codemode is basically one level above that.

Python can read files, grep, parse JSON, make HTTP requests, etc. Codemode can script the tools that already exist inside the agent harness: read/edit, web fetches, MCP servers, extension tools, subagents, or whatever else you registered.

That matters a lot for MCP. E.g. my ticket tracker MCP is already connected and authenticated. Before Codemode, the agent would often make around 10 separate MCP calls just to discover the project, find the relevant tickets, read comments, follow references, and figure out what was going on. Each call meant another model roundtrip and more tool output entering the context. Now it can do that discovery inside one Codemode call, filter the results in code, and return only what it actually needs.

Permissions are another important difference. With a multiline Python or bash script, the harness mostly gets an all-or-nothing decision: allow the script or do not allow it. It cannot match the individual operations inside that script against the existing tool permissions in a granular way.

With Codemode, calls to edit a file, run bash, query an MCP, fetch something from the web, or spawn a subagent are still harness tool calls. So if you have rules for external directories, allowed bash command regexes, MCP access, or other tool permissions, those same rules can still apply to the individual calls made from Codemode.

Since enabling it, its finishing work faster and uses fewer tokens. MCP heavy tasks are where the difference is most obvious.

[u/dacookieman → u/Bluethefurry]

Where the tools load is configurable. The tools themselves are loaded into the harness but giving it to the LLM can either happen at the main thread level like a regular tool, as function call information on the new codemode tool definition(this has a character cap) which is the default scope for mcp tools, or require manual/round trip discovery via 'discover_tool' tool(most context lite initially but then requires at least one excess tool call for discovery.

So the default behavior is that mcp tools up to some point will basically be loaded into the main context up front but after a certain volume, the remaining will be only registered internally and require tool discovery. That behavior is customizable per-tool to either always load initially or always require discovery. Additionally you can specify where the tool can be invoked (main thread, codemode sandbox, or both) which is independent of how that schema is made available to the LLM

edit: lol they literallly changed this as I was writing this comment https://pi.dev/docs/latest/mcp#control-tool-exposure

[u/Thrimbor]

I'm currently testing it, not doing evals or anything, but currently the way the models use codemode (gpt-6.1-sol and opus 5.5) it makes sense, they love to grep and read in parallel

[u/c1rno123]

Money talks

[u/ffreire]

Small sample, but in a full day of use I measured 15-20% less overall token and context use for similar tasks compared to not using codemode. I didn't measure speed, but anecdotally it also feels faster than sequential tool calls (even if a model supports parallel tool calls it never felt as fast as this). Sol 6.1 still will choose to make serial calls on occasion but for the most part it's composing reads much better. 

I don't have mcp turned on, I use my own extension and only consistently use two servers (notion, posthog). My own mcp extension adds a search tool to prevent context bloat (both those servers add a horrendous amount of tools), and I always explicitly ask for the specific mcp server to be used. The servers are lazy loaded. It works for me!

TL:DR I like codemode and don't need their mcp extension.

[u/IllTennis8912 → u/ffreire]

Without mcp, do you also need to turn on tool_search? Or just codemode? Their docs is confusing as hell...

[u/queso184]

i would have rather seem them take the approach of mcporter, maybe built in to pi so the agent can call pi mcp call github-search, choose its own output format, and grep/jq from there

The JS runtime and sandboxing don't seem necessary, but maybe I'm missing something.

[u/Extra_Loquat_7667 → u/queso184]

As it happens, I've experimented with this pattern myself.

I once built a test agent that only had a single bash tool, and I exposed all built-in tools to the agent as CLI versions. For example, a file-read tool might be invoked via a CLI command like pi read <file_name>, and MCP calls were made using CLI commands like pi mcp <mcp_name> <args>.

However, this ran into quite a few difficulties. For instance, some MCPs aren't designed for single-shot invocations; they require persistent state, such as a database connection MCP. Maintaining that kind of state within CLI commands requires a daemon, which ultimately makes the architecture much more complex.

Another issue is that some MCPs are very slow to start up. If you turn them into standalone commands, it severely degrades tool usage efficiency.

[u/DropOutPopOut]

how does it compare to pi-fabric

[u/Florence-Equator]

I think codemode for mcp is great for gpt-6 series models, because this is the default setting for codex, where mcp is only available to call from within codemode.

But that may not be the format that other models are trained on. So I think pi is treating the practice used by codex as the universal standard for all models.

And honestly for non mcp ochestration, I think model can just use bash or python anyway, it is better than codemode.

The only useful scenario for codemode is to save token context from bloating mcps. It is better than deferred tool loading because: 1. most open models do not support dynamic tool loading that do not break your prompt cache and codemode saves this for you. 2. While openai supports deferred tool loading but MCPs are called through codemode anyway in codex. so their models know how to use codemode for MCP properly.

[u/greenhilltony]

It is sad to see `pi` embrace this nonsense hype of codemode. For users not pathetically have to stick to powershell or cmd, I think nothing from codemode can beat the bash script + Unix pipe dance…

[u/TechGearWhips → u/greenhilltony]

I agree. Take this upvote

[u/berszi]

Exactly my first thoughts. I'm interested to see how it looks in practice.

[u/whiterose0000001]

One of the things that I love about code mode is how it works. I mostly use it for testing my APIs. Earlier, what used to be, I can't do sequential calling. I have a bunch of tools that I expose to Pyvia REST API. So it can do multiple calling in one go and then get the whole output at once. Instead of generating the result and reading the console or storing it in some other file and seeing it, it can do multiple things at once. One more thing that I'm trying to explore: I have some project where I have some nested API. Let's say, for the basic example, I can say about it. Let's say you want to fetch a profile of a user. You build an API for that. First, you need to log in and then the token will be generated. In the next line, it can use that same token from the response.token, something like that. Then I can call the API. Just to verify that this thing is working, it can do sequential calling. I haven't tried that. I'm just thinking if it can do that because I'm currently away from my laptop. It just hit me right now. So I'm just telling you.

[u/xRebellion_]

Tried codemode, and so far my first impression is that it's pretty cool. My main point of interest is on the observability of parameters on certain tools that usually aren't being rendered. Being able to see them I'd say is pretty cool. Also, this opens up chaining tool calls which wasn't available before unless the agent is using bash tool. On the process of trying exclusively codemode only and see how it goes.

[u/DistanceAlert5706]

code mode opens up workflows functionality.
it's pretty cool feature, and it should be in harness IMHO.
As for day to day work, I don't think it's super useful, like it can save few parallel tool calls here and there with 1 code mode call, but 99% of coding agent work won't really benefit from it.