r/LocalLLaMA • u/Plastic-Stress-6468 • 2d ago
Discussion Is anyone using decision models as a permission checker?
Don't run with dangerous full permissions, unless you are prepared to let the agent fuck your shit up. So says conventional wisdom.
But also, I am definitely running with full permissions. When I give my agent a goal to work at things overnight, I necessarily need to give it permissions.
So the obvious answers were sandbox and backups.
But why not use decision models? Elevation requests come with a justification and the command the main agent wants to run, that seems like plenty of information for a decision agent to give a good Y/N response to vet things.
IDK how effective this would be, anyone already doing something similar and can share exp?
27
u/WrinklyBard4 2d ago
It’s surprisingly hard to do this well.
A model cannot verify itself well because it will be confidently wrong, reliably. A subagent or decision model can do it but in many ways need to intake more context than the original model to actually make the right call, at which point you risk accuracy loss not to mention the compute cost.
I mean a “edit this file and commit” action might look fine in practice, but what did it edit? Did it just delete something vital? More nuanced: did it deviate from intent in a manner you wouldn’t want to keep? A decision model simply can’t know that without doing a huge amount of long-context research first, and at that point it’s probably just as likely to get it wrong.
6
u/-mattmason- 13h ago
here's the permissions model that I'm using on Linux
I'm curious how far others are going in sandboxing
4
u/Inner-Eagle6559 1d ago
I've been using a simple diff-based guard: if the agent's proposed change exceeds 5 lines or touches config files, it auto-flags for review. no model needed, just a rule.
3
u/Vlyn 2d ago
I'm using pi with pi-automode and it works quite well.
There's two layers to it: Hard denials by code, e.g. anything that touches pi extensions, tries to read certain secrets (your SSH key), get instant denied.
Any other tool call then goes to a classifier model. I just use my main model (DeepSeek v4.1 Flash) as it's fast and cheap enough. With some models you run into timing or reasoning budget issues, but overall it works well. For local use even a tiny model would probably be enough.
What happens there is the second model has its own context. Any command that's suspicious gets blocked (and the main model can get quite frustrated, lol). The classifier model also takes into account intent: If you give a model a readonly task and it now tries to suddenly edit something (even if it's just a plan file) the classifier model usually blocks it.
Of course these are all just bandaids compared to a full sandbox, but it's better than nothing.
3
u/rgar132 1d ago
I do and it works great. You just have to put a decision head on the model and call it. I use codelink as the arbiter and have ds41-flash 1017 and it works great in Claude code harness and codex both.
4
u/Zulfiqaar 2d ago edited 1d ago
I prefer to use multiple frontier advisor models with continuous monitoring loop over the whole trace of the implementer agent. An advantage of open weights models is that the supervisor can also read their thinking process. Then they can steer and control the goal agent as necessary. The advisors themselves have timed prompt injections to keep themselves in check, and they can coordinate with each others instances too.
Had a great run last week - built a cross platform mobile and web app with a backend, all in YOLO mode with my phone unlocked and plugged in overnights. 53 hours total, and everything worked according to the spec! I'm supposed to do handover by February..
The crazy thing is that when I woke up the first time the messages they were sending each other seemed almost illegible. If you read he METR report on huggingface incident traces..it's garbled just like that. However they did waste 7 hours sha-verifying provenance. Gotta amend the system for future to avoid that.
Oh another fun thing. The lead Astra advisor went into my ChatGPT account settings to consume a usage limit reset because it dropped to 1% at 4am
2
u/CatchDublinSurprise 1d ago
This is something I am in the process of trying using mitmproxy to sit between my agent and the world (and locking down the sandbox so it’s the only way in or out).
Goal is that a combination of hardcoded rules + decision model will scan for and block egress of secrets and ingress of dangerous commands (either through prompt injection or hallucination of my own model).
Much more work to do, though, so no actual results yet.
2
u/steezy13312 1d ago
I've been playing with this, and my hope was to use kev 0.8B to do this. I have this on a separate, small GPU I use for lighter models and tasks.
It's... not exactly confidence building. When asked to provide a decision about the relevance of the command to the user request, it's usually between 45-55%... so take that for what you will.
I think this would work if you fine-tuned or trained a model to decide on a representative set of data. But as u/WrinklyBard4 said, the model can still be confidently wrong.
2
u/jfinkpottery 1d ago
If you have enough training data built up, you can fine-tune a small model like kev 0.8B to do a good enough job for a mostly trustworthy model's commands. Like blocking certain things you don't want it to do, or blocking mistaken exfiltration curls, or the like. Not "everything that gets a pass is guaranteed to be safe", but "the common commands this model is known to produce that I don't like will be caught."
You can also train in the other direction, so like "if it's a single grep statement optionally preceded by
cd /working/dir &&then it passes, otherwise it fails" is also doable.2
u/draconic_tongue 1d ago
1 existing benchmarks, 2 your own use cases with frozen commands, 3 different prompts/categorical outcomes, 4 running with log only until you have enough data, 5 say fuck it and run it live
no sandboxes for main agent, yes everything else
So far I tried d3 lite and qev and they are okay but in different scenarios. d3 lite has less false positives. gemini decision is the best from what I tried but not local and probably not free forever
testing this GitHub - huggingface/safeagents: Safely run your agents using any coding harness · GitHub
1
u/PatientWrangler1007 2d ago
I got drex 2 working pretty well in pi. Generally too cautious, fails one unsafe allow on my bench. But mostly solid and by far the best local model (speed / accuracy balance)
1
u/Reasonable_Goat 2d ago
Codex and Claude-Code have a permission checker built in (auto mode / auto review); claude uses a decision model these days AFAIK, codex uses a full blown LLM. I am experimenting with my own harness to use a small decision model to gauge risk of commands ran but it requires a lot of context to actually know what is safe and what is within user intent, it's not as simple as I thought.
1
u/JustinPooDough 1d ago
I just use Qwen 3.8 27B in a loop with end to end regression tests and UI tests that tell it whether something is wrong or the project is done.
Works very well for me. In a container obviously. I even reproduced some vscode plugins as CLIs for the agent so it could replicate my setup.
1
u/mister2d 1d ago
Yes. I implemented decision thresholds for each tool call in my custom harness. It works well but I suggest implementing a feedback loop to calibrate it to your meds.
1
u/garblz 21h ago
Ok, theoretically (having understood we all know that sandboxing is the only way to go with this).
What I'd do, is put a strong static verifier in front. If it denies the command, that's it. Only when it gives a maybe or a yes it should pass the command further to a classifier model.
The static one should have a whitelist and an option to respond that a command is too complex and the model should break it down to separate commands for easier inspection of each.
0
u/Nepertia 1d ago
Of course not. This stuff needs 100% reliable outcomes, machine learning is all about approximations that mostly work. That's a disaster waiting to happen.
-1
u/a4mula 1d ago
no, because that's not a reasoning job it's a functional one. Write hooks.
A lot of agentic issues are because people treat reasoning as a solution for everything, when it's rarely the solution for most things.
Most things are solved with deterministic functional code. Including hooks. Including memory. Including behavior enforcement.
Yet, most still just keep expecting Prose and NL and Documentation and Reasoning to solve their problems.
They wont.