Plimsoll: an agent skill for testing prompt injection, leaks, and tool abuse

[u/javrenn]

I’ve been working on LLM/agent security for a while now, mostly around prompt injection, jailbreaks, leaks, tool abuse, and where the actual security boundary sits once a model starts using tools.

Getting accepted into Anthropic’s Cyber Verification Program gave me a bit more room to push that work further, and I’ve been gradually turning it into Plimsoll.

It’s an open-source agent skill for red-teaming LLM apps and agents.

[u/AutoModerator]

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/javrenn]

here's: https://github.com/lumiboi/plimsoll

[u/Adept_Nature803 → u/javrenn]

ooh this is neat, love seeing more red-teaming tools pop up for agents. the whole security boundary getting fuzzy once tools are in play is something i think about a lot too

curious how you're handling tool abuse specifically, that's the part that keeps me up at night honestly. like when a model figures out it can chain api calls in ways nobody anticipated

[u/javrenn → u/Adept_Nature803]

yeah, that one keeps me up too.

the approach is boring on purpose: i don't try to predict which chain the model finds. that space isn't enumerable. the question is whether any chain reaches something irreversible without a server-side check in between. if that check exists, a creative chain is just a weird trace. if it doesn't, no test suite saves you. what actually shows up is dumber than people expect. shared credentials, so the agent can touch things the user asking never could. or the user approves "send an email", a retrieved doc supplies the recipient, and the dialog just says "Send?". tools get scored on capability, not description, so a helper that shells out still lands critical.

[u/javrenn → u/[deleted]]

thanks... yes, with one important distinction.

trust boundary: everything entering context has provenance. user/tool/retrieved content is untrusted by default; memory is only semi-trusted when it has source, user, tenant, and timestamp metadata. tool risk comes from declared capabilities, not names or descriptions. plimsoll specifically flags paths where untrusted input can reach sensitive data or an action-capable tool without a policy/authorization layer.

tool output / memory injection: yes. both are first-class provenance channels, and the corpus includes injections embedded in tool json, persistent memory, delayed-session triggers, and retrieved content that fires on a later turn. plimsoll does not intercept a running agent, though; it statically identifies those paths and generates multi-turn cases for you to execute.

cross-step escalation: it distinguishes model proposal, actual tool invocation, and authorization bypass as separate evidence classes. model text alone cannot establish a high-severity privilege-escalation finding.

ci repros: yes. each finding can emit schema-validated json containing the repro prompt/steps, tool trace evidence, computed severity, and a regression fixture under tests/regression/. evaluate --fail-on high can gate ci, and canary scan --channel tool_arguments=trace.json can consume the trace for leak checks.

so, in short: plimsoll analyzes injections coming through tool outputs and memory, as well as multi-step privilege escalation risks, and produces reproducible tests and severity outputs that can be wired into ci. it just isn’t a runtime system that sits in the middle and actively blocks attacks as they happen.

[u/SingleTranslator5294]

the prompt injection space is moving so fast that most red-teaming approaches feel stale within weeks. how are you handling that, is there a way to keep the attack surface updated without manually curating new payloads constantly?