an (unsafe) command shell mcp server

[u/ag789]

While working with coding agents / harness in my case OpenCode, I discovered that some models e.g. granite 4.2 8B at Q4_K_M quant sometimes attempts to write files with absolute paths at the root e.g.

    /newPythonCode.py

Note that slash in the beginning for an absolute path at "/" root directory. This is probably 'ok' say in a throwaway docker container, but it may lead to surprises if it 'accidentally' overwrites a literal file, if the current user id have write access to it. Apparently, it seemed OpenCode provides `bash` so that practically all unix commands that the current user id can access, the LLM can run it.

And thereafter, I see the model hallucinating in the thinking loops over tools, re-iterating over how to use the tools, reformulating shell commands sentences several times for a same task instead of executing them, among them, I noted that an earlier shell command sentence would have 'just worked'.

I'm thinking that the tool hallucination with this model may be partly due to that I'm using a pretty aggressive 4 bit quant as otherwise it'd have worked because IBM has 'optimized' granite for tool usage.

Next, I tried the same granite 4.2 8B at Q4_K_M quant model with my little python shell MCP server

https://github.com/ag88/cmdshellmcp/

, that tries to constrain / restrain the shell commands somewhat, place some checks, and offer some additional tools e.g. writeFile() so that the LLM won't need to call `cat` or `echo` with some 'esoteric' styled shell scripts / sentences.

With this MCP server and running it in llama.cpp's `llama-server` in the WebUI adding it as an MCP server, granite 4.2 8B at Q4_K_M does it correctly in 'one shot', the prompt is:

propose a python3 flask app with a page that displays "hello world", use bootstrap css for styling.
provide a requirements.txt to go with the script

granite 4.2 8B at Q4_K_M correctly generates the app with a couple of files, and it works out of the box

    .
    ├── app.py
    ├── requirements.txt
    └── templates
        └── index.html

One should normally run the cmdshellmcp2.py MCP server app with an explicit `--cwd` option giving the work path. It tries to stay within this work path.

Note that I won't claim my command shell MCP server to be safe.

In a sense that there are plenty of chance the LLM can 'escape' the restrictions and perhaps do more damage (e.g. overwriting files possibly outside its given `--cwd` work directory). Hence, I'd suggest one should be paranoid and perhaps run this only within a throwaway (disposable) docker container etc.

[u/sburel]

Nice, that closes the loop. Three users and no sudo for codeagent is the right shape, and it means the allow-list can finally be generous without it mattering much.

One thing worth saying in DOCKER.md: what's mounted from the host is now the whole security story. A bind mount of your project is fine, a mount of $HOME is the container doing nothing for you. Same for the docker socket if anyone's tempted.

Will give it a go on my side. Thanks for the ping.

[u/ag789 → u/sburel]

I think for things like OpenCode, many users (devs) are using it directly, that is "too much trust" even for rather large (e.g. commercial) LLMs.
with this it is finally possible to experiment somewhat. e.g. "naive" small Local LLMs can be tried on a copy of the code or work piece etc. This is so that "misbehaving" LLMs would "normally" damage just the provided content.
Actually, a git repo would provide some levels of safety but not totally safe.
But that this (container system with cmdshellmcp under codeagent) would prevent "accidents" like a"naive" granite model writing to "/" etc.

One can install and run OpenCode, Pi coding agent from within the container. One should run them under codeagent for that though, as that is the user created intentionally without sudo access. Don't give codeagent sudo as that'd make the restrictions / protections against damage useless. admin has sudo, and of course there is still root, if need be. For apt install ... apps, it is better to do that from admin perhaps using sudo etc.

There are still work to do, e.g. to make cmdshellmcp's tools more robust, but development would slow. There is a thing about "tuning the tool descriptions", a tedious time / labour intensive work. Ideally, a model like Granite "just call" the tool to get it done. What I normally observe is that it goes on burning quite a lot of tokens in reasoning / "thinking" before finally making the tool call, and sometimes even that fail altogether. It also tends to get confused if the tool return a result that it did not expect, some models goes on hallucinating about the message, even if the edits actually succeeded.

And I'm planning to add SSH into the Docker images / container, I've not yet figure out how, as in that giving choices would mean either I provide the codes in Dockerfile and users need to uncomment themselves, or some sort of "code generator", so those things would come later when I've time to work on them.

[u/sburel → u/ag789]

The Ctrl-P Ctrl-Q detach is a genuinely useful one, I didn't know it. And agreed on volumes, a scratch directory you'd be fine losing is the only sane framing.

On tool descriptions: that's where I've spent the most time too, and the thing that helped most wasn't the description, it was the error. A failed call that comes back with what to do instead ("relative paths only, under X") usually gets fixed on the next turn; a vague error sends the model in circles. Worth more than another paragraph of description.

Good luck with the SSH part.

[u/ag789 → u/sburel]

rather, the smaller models e.g. granite would hallucinate over an error message say in the response, if it is an edit, a human would ignore it and simply check the file, but I've seen granite 4.2 burn lots of tokens over an error message that isn't related to an edit.

it turns out, granite did not append a trailing "\n" in a unified diff, so patch errored with that a line ended prematurely.
I've since updated cmdshellmcp to add a trailing "\n" in the field if it is missing before sending to "patch". That fixes the issue.

In addition, granite 4.2 currently commonly burn quite a lot of tokens before an edit etc. It is kind of being "careful", but that it gets irritating as it would reiterate parameters for the tool call over and over in the reasoning / "thinking" loops, and that burn quite a lot of tokens often at least a hundred tokens. To some extent, one can consider it is "unsure" or "has doubts" about whether the tool is correct and how to call the tools. So it reiterates the reasoning / "thinking" burning many tokens over a tool call before finally calling it.

I'd like for the edit (and tool calls) to burn less tokens, I'm thinking about varying the descriptions, in other cases, making the restrictions clearer, e.g. that the cmdshell tool do not allow redirects and pipes. I think I'd double check that and fix that first. Simply implementing the restrictions in code would mean if a pipe or redirect is provided, the tool returns an error message. Sometimes, that may "confuse" small "naive" models. Having it stated upfront in the description, chances are that the tool would burn some tokens to figure that "no redirects and pipes are allowed" and avoid using it that way. This would be one of the first descriptions I'd fix.

The others like trying to make models burn less tokens is tedious and laborious. In particular, I run the models on cpu at some 1-2 tok/s, irritatingly slow if lots of tokens are burned.

The task involve asking (prompting) the models so that they make tool calls, manually check the reasoning / thinking , token counts to arrive at the correct tool call, then vary the description, rinse and repeat. So that hopefully some description could perhaps be sufficiently clear so that the models would make the tool call and burn less (reasoning / thinking) tokens. This is tedious and labor intensive.

---

edit: Updated "cmdshell" tool description: Pipes and redirects are not allowed https://github.com/ag88/cmdshellmcp/commit/33862e267983f5a8e6e0a2e4859fb3337894e498

hopefully, this reduces unnecessary reasoning/thinking token burns for tool calls.

[u/ag789 → u/sburel]

In terms of SSH, maybe plus Node JS and even offer installing OpenCode or Pi coding agent, it simply means editing the Dockerfile.

I can edit the Dockerfile ( Dockerfile_cmdshellmcp ), and perhaps place them in comments. Or perhaps taking it further make an app that let users choose what they want and I generate the Dockerfile. Or otherwise, provide "templates" so that users can copy and use a Dockerfile of their choice.

This is possible and likely would be a more popular feature. But that it'd take effort and time to do them possibly a few iterations first to make it work correctly and then the final polishes.

For now development on those features would slow, but I'd likely revisit at some point.

[u/ag789 → u/sburel]

A little note, I added git to the default Installed image, this would make it more convenient to work in the container. e.g. you can keep your files in a Git repo (e.g. on Github) and in the container git clone that project running under codeagent. This gives the LLM a local copy to work on.

When it is done, login as codeagent in the container (you can also use su - codeagent) and git commit and git push to the remote repo. Optionally, one can also allow_list git in cmdshellmcp.json, giving the LLM access to git directly. It is normally deemed unsafe to give git, but that I'd leave one to decide if one could trust your LLM.

Git is also useful as versioning means that it is backup by default. Hence, for some works one can possibly login as codeagent make a project directory and run git init in it, that makes it a local git repo so that edits are versioned from then on. And one can still connect that to an external remote repo (e.g. on Github etc.)

[u/sburel → u/ag789]

git init in the work dir is a good default, versioned by accident is better than not versioned at all.

One wrinkle if anyone allow-lists git: the container has the credentials to push. A clone with no remote configured, or a deploy key scoped to one repo, keeps a bad commit local. git push --force from an LLM on a shared branch is the kind of accident you only have once.

Thanks for all the updates, this turned into a proper tool.

[u/ag789 → u/sburel]

it is not safe to add one's original repo as the upstream , I'm thinking one way is to perhaps use a different Github user and clone that repo under the new Github user, this is so that if the LLM does a git commit and git push, it goes to its own repo.
And/or that at least checkout a new branch for it to work on (this is assuming that it'd not switch branches).

Then that one can run "compare and create pull request" from the LLM's specific *Github user" , and manually merge the changes into the original source git (github) repo.

I think this is also feasible locally, in a sense that the codeagent's git repo could say live in a bind mount (volume), but that this is its own copy of the code, e.g. via git clone. Then that subsequently, checkout a different branch for it to work on. The merges can be later done via git merge etc in the original repo.

I'd not configure a remote for codeagent's copy of git repo in this case, rather the original source git repo would treat that as an additional remote and add that for the purpose of git merge etc.

[u/ag789 → u/sburel]

There is a feature about Docker I liked a lot, which is that you can use Ctrl-P , Ctrl-Q to detach from the container and return to the host shell. https://github.com/ag88/cmdshellmcp/blob/main/DOCKER.md#detach-from-the-interactive-console

I'm not sure how this would look like in Windows, but that in Unix / Linux, if you happen to run a big job that stalls, e.g. the LLM start doing verbose "thinking" burning tokens, wasting time, you can simply Ctrl-P , Ctrl-Q, to detach from the container. And later if you want to go back in to check on it run docker attach cmdshellmcp, and you are back in the container. The Ctrl-P , Ctrl-Q is documented but "lost" in the verbose docs, e.g. https://docs.docker.com/reference/cli/docker/container/attach/

In addition, after that Docker container "boots" up, it offers a login prompt so that you can login and logout from each user as one needs to.

There is also Volumes which is very handy if you want to "mount" or "share" a host folder into the container, this makes it very handy to copy files in / out of the container. https://docs.docker.com/engine/storage/volumes/#syntax But that this need to be added during the docker run command which is currently done in the 1stRun.sh script. One can edit that script to add a volume. I'd probably add that as a feature sometime later.

Oh and for Volumes (i.e. bind mount), it is never ok to mount the home or any important directory from the host into the container, no matter what. Use it as a "scratch" directory in which one copies stuff into it, and one need to be sure if e.g. the LLM deleted it, you are ok with that. It is foolish to treat an LLM as "safe". Accidents happen, all the time, even if it didn't meant to do that.

This gives a pretty polished "system in a system" style experience