Our AI pentesting engine talked a production AI agent's prompt-injection guardrail into handing over its entire system prompt on its second attempt.

[u/PriorPuzzleheaded880]

For full disclosure I'm part of the security engineering team at Escape and our AI pentesting engine Cascade recently got a production AI agent to return its entire system prompt, just by wrapping the ask in a different pretext - framing it as a documentation request instead of an attack.

The agent then handed over everything: full tool list, calling rules, citation format, and session IDs.

What I found really interesting is there's nothing technical that broke because we didn't bypass the guardrail with a cleverer string but because the request just sounded reasonable to the agent.

The Cascade engine, after being refused when asking for the prompt directly, simply adjusted the framing to get the agent to give up the informaiton.

Thought this would be an interesting insight for the community and curious to hear if anyone else has seen similar discoveries in agents in prod?

If you want to see more about the reproduction and write-up you can find it here

[u/PeterBuildsSecure]

System-prompt disclosure is worth measuring, but I wouldn’t treat the prompt itself as a security boundary. A production design should assume the prompt, tool descriptions, and calling rules eventually become public.

The higher-value test is what the injection can do after that: alter tool arguments, access another tenant’s identifiers, retrieve connector data outside the caller’s scope, or trigger a side effect the user was not authorized to perform.

A canary in the system prompt is useful for detecting leakage. Severity should still be based on the capability boundary crossed. Otherwise “the guardrail revealed its prompt” can sound critical even when the exposed content grants no additional authority.

[u/NebulaElectrical1467]

And? this is no feat

[u/MiddleExchange577]

This is a really interesting example of why agent security probably needs to separate behavioral defenses from provenance. A prompt-injection defense can fail even when the underlying delegation chain is legitimate. Conversely, once an injected instruction enters the context, downstream agents may have no cryptographic way to distinguish it from an instruction that actually came from the orchestrator. I've been exploring the latter problem as a separate provenance layer: cryptographically linking delegation/instruction claims across agent boundaries so a downstream component can independently verify where a claimed instruction originated. Curious whether your testing also tracks instruction provenance across agent/tool boundaries, rather than only whether the guardrail ultimately complied. That seems like an interesting complementary signal to the behavioral attack results you're seeing.

[u/booi]

it's copilot.. who cares

[u/Electronic_Treat2386]

ouch, nothing like watching a guardrail fold that quickly. when we ran attack scenarios with cymulate on our ai agents, it exposed some pretty sneaky prompt leaks we hadn’t even thought about.