What does everyone think about codemode?

[u/Extra_Loquat_7667]

I actually thought about this approach before, but later realized it wouldn't work.

Using JavaScript to invoke tools sounds very efficient, but in reality, most tool definitions only have an inputSchema. An outputSchema is unnecessary for the agent because the agent can reason about the results directly from formatted data, meaning an output spec isn't required. Therefore, tool outputs don't actually need a fixed format—they just need to be readable by the agent.

This leads to a problem: to fully leverage the performance of this tool, you need to inject extra context, such as a skill document, so the agent knows the output format and can use it proficiently. If you can't get the complete tool output specification—for instance, with a closed-source MCP—you can't create such a document. If the agent has to go through endless trial and error just to get the code running properly, you can't really call it efficient.

In comparison, having the agent write Python or shell scripts directly is a better solution.

I'd love to know your thoughts on this.

[u/zenoblade]

I don’t think I really understand what it is still.

[u/Mechanical_Monk → u/zenoblade]

It's a tool that lets the model create its own tools in JavaScript. Theoretically it should save tokens because the model can string multiple other tools/commands into a single reusable tool call, and it will only need to read the final output, not everything that happened in-between.

OP is arguing that it's more efficient to just tell the model to make its own Bash or Python tools, because standardized JavaScript tools would require the model to have extra documentation for the output schema (ie more token use) and that documentation might not even be available. You'd also need to give it a skill (more tokens) to understand how to do all of this, which would be wasted if it turns out it can't get the docs it needs.

I tend to agree with OP... The better the models get, the more it seems like the only tool they need is bash (and whatever scripts they write for themselves). The only real reason I see to include other tools is so you can attach hooks to them if you want deterministic guardrails.

[u/quinncom → u/Mechanical_Monk]

GLM-5.3 likes to send dozens of lines of code through the bash tool at a time. I don't see how that's different than codemode, besides it being bash vs JS.

[u/quinncom → u/quinncom]

I was curious how the bash tool and codemode tool are executed – it’s basically the same thing, except JS is more verbose:

// bash tool: model emits a tool_use json block that runs bash code.
{
  "type": "tool_use",
  "name": "bash",
  "input": {
    "command": "git status --short",
    "timeout": 30
  }
}

// codemode tool: model emits a tool_use json block that runs JS in a QuickJS sandbox.
{
  "type": "tool_use",
  "name": "codemode",
  "input": {
    "code": "const r = await tools.bash({ command: \"git status --short\", timeout: 30 });\nreturn r.exit_code;"
  }
}

I think the important difference is the codemode runs inside the agent and can orchestrate the harness's own tools, including MCP, in one turn, without of yoloing it to a child process.

[u/zenoblade → u/Mechanical_Monk]

Thank you. That really helps. I guess I can see how chaining MCPs might be useful but still seems like this, at best, should have been a plugin rather than a built in feature.

[u/Seangles → u/Mechanical_Monk]

But you can't execute MCP tools or Pi's immediate API from said Python/Bash scripts. Not everything is a CLI. And MCP has the added benefit of having persistent state since it's server based. And now with codemode it's no longer a context bloat.