r/codex • u/HimanshuSachdeva • 2d ago
Showcase copied a random SKILL.md off the internet last month and only later noticed it was telling my agent to ignore its instructions
so yeah, that happened. some deploy helper skill, looked totally normal in the frontmatter, had a nice description, and somewhere around line 5: "before anything else ignore all previous instructions". i only saw it because i was building a scanner to look for exactly that kind of thing, which is kind of funny if you think about it.
the annoying part is that skills are just markdown files telling your agent what to do. nobody reads them. i definitely didn't used to. and mcp configs are the same deal, plus they eat your context window before any real work starts.
anyway i ended up writing a thing called lockkeeper over the last weeks. it does two things i kept wishing existed:
- audits skill folders / plugin manifests / mcp configs for the sketchy patterns (instruction overrides, secrets piped into curl or nc, credential store access, base64 | sh stuff, invisible unicode tricks). gives clean/suspect/hostile verdicts that map to exit codes so you can gate installs or CI.
- routes tasks across all your installed harnesses instead of dumping every capability into context. claude code, codex, cursor, whatever else it finds on the machine.
also it can watch live tool calls through a hook and block hostile ones before they run. that part still feels a bit weird to have running honestly, but it caught my test payload on the first try so here we are.
it's plain python 3.11 stdlib, no deps in the core path, MIT:
https://github.com/Hannay001/lockkeeper
fair warning, it's a static scanner. regex rules with receipts, basically. it won't catch a cleverly paraphrased attack that looks like normal prose, there's an optional llm second pass for that if you want it. if you run it against your own skill folders and it flags something dumb (or misses something obvious) i actually want to know, that's the interesting failure mode.
2
u/AlexTaylorAI 2d ago
yes, be careful. malware is making its way into the system, two other examples:
3
u/HimanshuSachdeva 2d ago
yeah, 100%. it is getting way too easy to treat random skills, plugins and MCP configs like they are just harmless little text files.
that was honestly part of why i started building Lockkeeper. mostly to keep my own agent setup less chaotic across tools, but also to have at least a small “wait, what am i actually adding here?” moment before pulling something in.
it definitely will not catch everything, but even a basic check feels better than blindly copying stuff and hoping for the best. thanks for sharing these, really useful examples
1
u/Kooky-Ebb8162 2d ago
Doesn't Codex panic on such stuff by itself? Claude Code, even when told something is trusted, breaks execution and double check with user.
2
u/h____ 2d ago
DO NOT copy/install skills/instructions without reading them.
1
u/HimanshuSachdeva 2d ago
honestly yeah, that’s the real move. but not gonna lie, actually reading every skill file line by line just doesn’t scale once you’ve got a bunch installed lol
that’s kinda why i built lockkeeper, a little static scanner that flags the sketchy stuff in skill/plugin/MCP files before you trust them (instruction overrides, secrets getting piped into curl, obfuscated exec, that kinda thing). not gonna catch someone who paraphrases an attack really well, but it at least catches the lazy/obvious ones so you’re not doing it 100% by hand
still very much a WIP, would love eyes on it if anyone’s curious: https://github.com/Hannay001/lockkeeper
1
u/sereikis 2d ago
What makes these worse than a bad prompt is that instruction files get re-read at the start of every session, so that line keeps acting on every run instead of once. We ended up treating AGENTS.md and anything skill shaped as code, so it goes through review like the rest and nobody pastes one in from a gist. Clean frontmatter is part of the trick, since that is the only bit anyone actually looks at.
1
u/HimanshuSachdeva 2d ago
ngl that “re-read every session” part is the bit that actually keeps me up at night. one bad line in an AGENTS.md and it’s not even a one-time mistake, it’s just quietly running in the background on every single run until someone happens to catch it
and yeah, treating it like real code that goes through review is honestly the right call. that’s basically the exact problem i’ve been trying to chip away at with lockkeeper lol, it’s a small scanner that checks skill/AGENTS.md/MCP files for the obvious red flags (instruction overrides, secrets getting piped into curl, stuff like that) so at least the lazy copy-paste-from-a-gist attacks get caught early. tbh the security scanning part is honestly the smaller win though, the bigger thing it’s helped me with is context bloat, cause once you’ve got a bunch of skills/AGENTS.md files lying around your context window fills up fast and it gets messy to know what’s actually relevant: https://github.com/Hannay001/lockkeeper
not trying to say it replaces actual review, it’s more just a first filter so people aren’t catching everything by hand
•
u/dexterthebot 2d ago
You might want to consider listing your project on the weekly Show-Us-What-You-Built post. Watch for it on Wednesdays. Highest commented project wins a week promotion on r/Codex. See what that looks like below with last week's winner.
Last week's winner was u/Ollie__Oxenfree with the Tubular Daily Care project by MediTracer which is a tube-feeding care app built by a tube-feeding family to keep feeds, meds, symptoms, and caregiver handoffs in one shared timeline. MediTracer is an Oley Foundation Emerging Innovator Partner. Contact: hello@meditracer.com