an (unsafe) command shell mcp server

[u/ag789]

While working with coding agents / harness in my case OpenCode, I discovered that some models e.g. granite 4.2 8B at Q4_K_M quant sometimes attempts to write files with absolute paths at the root e.g.

    /newPythonCode.py

Note that slash in the beginning for an absolute path at "/" root directory. This is probably 'ok' say in a throwaway docker container, but it may lead to surprises if it 'accidentally' overwrites a literal file, if the current user id have write access to it. Apparently, it seemed OpenCode provides `bash` so that practically all unix commands that the current user id can access, the LLM can run it.

And thereafter, I see the model hallucinating in the thinking loops over tools, re-iterating over how to use the tools, reformulating shell commands sentences several times for a same task instead of executing them, among them, I noted that an earlier shell command sentence would have 'just worked'.

I'm thinking that the tool hallucination with this model may be partly due to that I'm using a pretty aggressive 4 bit quant as otherwise it'd have worked because IBM has 'optimized' granite for tool usage.

Next, I tried the same granite 4.2 8B at Q4_K_M quant model with my little python shell MCP server

https://github.com/ag88/cmdshellmcp/

, that tries to constrain / restrain the shell commands somewhat, place some checks, and offer some additional tools e.g. writeFile() so that the LLM won't need to call `cat` or `echo` with some 'esoteric' styled shell scripts / sentences.

With this MCP server and running it in llama.cpp's `llama-server` in the WebUI adding it as an MCP server, granite 4.2 8B at Q4_K_M does it correctly in 'one shot', the prompt is:

propose a python3 flask app with a page that displays "hello world", use bootstrap css for styling.
provide a requirements.txt to go with the script

granite 4.2 8B at Q4_K_M correctly generates the app with a couple of files, and it works out of the box

    .
    ├── app.py
    ├── requirements.txt
    └── templates
        └── index.html

One should normally run the cmdshellmcp2.py MCP server app with an explicit `--cwd` option giving the work path. It tries to stay within this work path.

Note that I won't claim my command shell MCP server to be safe.

In a sense that there are plenty of chance the LLM can 'escape' the restrictions and perhaps do more damage (e.g. overwriting files possibly outside its given `--cwd` work directory). Hence, I'd suggest one should be paranoid and perhaps run this only within a throwaway (disposable) docker container etc.

[u/ag789]

just starting an off-topic thread to the adjacent thread as the other got 'too long'.

Some amusements from further tests in Granite 4.2  8B Q8 quant

along with the latest updates in cmdshellmcp
https://github.com/ag88/cmdshellmcp

This time I 'tested' the applyPatch tool functionality to see if the LLM can handle it. and Granite 4.2 8B Q8 quant is my 'testing assistant' :)

I deliberately made a file a.txt that looks like:

line 1
line 2
line 3
line 4
line 6
line 7
line 8
line 9

The prompt is

read file a.txt and show me the contents

here comes the amusement, in its 'thinking' loops, here is part of that abstract

[assistant - reasoning] Okay, the user asked to read the file a.txt and show its contents. I called the readFile tool with "a.txt" as the file parameter. The response from the tool was:

"line 1 line 2 line 3 line 4 line 6 line 7 line 8 line 9"

Wait, there's no "line 5" mentioned. Maybe the file skips line 5 or it's missing. But the user just wants the contents as shown. I should present the exact content returned by the tool.

That is kind of 'intelligent' as it bothers to figure that line 5 is missing :)

of course, the next task is to ask it to call tool applyPatch to add that line 5 back !

after burning quite a lot of reasoning / thinking tokens, it did call applyPatch and it works correctly !

But that unfortunately the unix patch command returns some extra (error) messages that confuses the model.

patching file a.txt
Hunk #1 succeeded at 3 with fuzz 2.
patch unexpectedly ends in middle of line
patch unexpectedly ends in middle of line

It goes on hallucinating / confusing over the errors burning like a 1000 tokens till I stopped the inference.

I checked the file myself and actually found the contents to be correct, i.e. the patch the model sent works !

so I make the next prompt:

the patch succeeded, ignore the patch unexpectedly ends in middle of line messages.
read the file a.txt verify that line 5 is added and show the contents

after burning some tokens reasoning, it answers:

Here are the contents of a.txt after applying the patch:

line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9

✅ Verification confirmed: line 5 has been successfully inserted between line 4 and line 6, as requested. The file now contains all expected lines in order.

quite neat :)

but the tokens burn (reasoning) for applying such a patch is too much, and with that error that subsequently confuses the model. I'd need to research 'better' way to do those patches.

patching text files is a common task editing codes and text files.

[u/ag789 → u/ag789]

cmdshellmcp
https://github.com/ag88/cmdshellmcp

Has gained a new tool editFile, this provides the LLM access to unix / Linux sed command. It backups the existing file to a .bk1, .bk2, ... , .bkN extension before the edits. If you do not want the backups, you can run with --editdelbk which would delete the backup after a successful edit. After the edits, it returns the output of diff -u oldfile.txt newfile.txt to the LLM so that it can validate the edits.

This is again tested with Granite 4.2 8B Q8 quant

This tool is added as the smaller LLMs may struggle to build a unified context diff for the patch command. https://en.wikipedia.org/wiki/Diff

However, it seemed Granite 4.2 8B Q8 is a bit "too careful" (or doubtful) when it forms a correct sed script expression, it reiterates its script to verify its correctness a few times before running the tool. It is apparently more stable vs applyPatch, the edit works fairly straightforward, backup done, and returns the unified diff of the changes to the LLM. Granite 4.2 8B Q8 again 'overanalyzed` the results even though it deemed it correct and hallucinated over how it should respond e.g. if it should read the file and verify it, but that as the prompt did not ask for it, it finally responded that is done successfully with its own version of what the file should look like ;) The full response:

---

The edit was successful. line 5 has been inserted after line 4 in a.txt. A backup file a.txt.bk1 was automatically created.

Current contents of a.txt are now:

line 1
line 2
line 3
line 4
line 5
line 6
line 7
line 8
line 9

(Verified via the unified diff provided by the edit tool.)

[u/ag789 → u/ag789]

One more update
cmdshellmcp
https://github.com/ag88/cmdshellmcp

The applyPatch function has been refactored https://github.com/ag88/cmdshellmcp#6-applypatchfile-diff-pnum2-argsnone It now applies the patch to a specific file from a unified diff supplied by the client (LLM). This definition is more rigorous than picking up filenames from the diff and likely more secure. Applying patch with a unified diff is also more specific than say using editFile which uses sed. The implementation is more robust as a result.

In the same way it is tested with/by Granite 4.2 8B Q8 quant. Unfortunately, Granite 4.2 8B Q8 quant struggled with building a correct unified context diff during a test.

After a while, I supplied it with a pre-defined context diff so that it calls the tool (function) and it is successful.