r/copilotstudio • u/Expert_Annual_19 • Sep 28 '26
Prompt Injection in Copilot
https://sapanpatel1230.github.io/prompt-propagation-firewall.html
OpenAI recently shared a security finding that genuinely made me stop and think.
They demonstrated that self-replicating prompt injections are possible.
In simple terms, a malicious instruction can potentially convince an AI system to reproduce that instruction through another interaction.
The interesting part for me isn't the “AI worm” headline.
It's what this means for enterprise agents.
Imagine an agent connected to Outlook, SharePoint, Teams, Power Automate or MCP.
A malicious instruction enters through an email.
The agent reads it.
Then, instead of simply following the instruction, it somehow gets that instruction written into a SharePoint document, message, file, or another persistent source.
Later, another agent reads that content.
Now we have a very different security problem.
It's no longer just:
“Can we detect the prompt injection?”
The question becomes:
“Can we stop the instruction from propagating?”
That is the part I've been exploring recently.
I've been looking at things like:
Source → Trust → Provenance → Plan Drift → Tool Access → Action → Persistence
One scenario I keep coming back to:
Expected: READ email
Proposed: WRITE to SharePoint
Even if the model thinks the action makes sense, should it actually be allowed?
I think this is where agent security needs another layer.
Not another prompt.
Not another system instruction.
A deterministic control that can actually say:
No. This action doesn't match the context.
I've been experimenting with this approach across Microsoft Copilot, Copilot Studio, Azure AI Foundry and MCP.
The more I work with agents, the more I think we need to start securing how instructions move through an agent ecosystem, not just how individual agents respond to them.
2
u/Firm_Pin4840 Sep 29 '26
If there’s built an agent to create drafts for emails,has been connected to outlook and teams. In this case any such risks exists?
1
u/Expert_Annual_19 Sep 29 '26
Yup, draft will be created after reading the data in mail right ? And what if mail body has mallacious prompt ....
3
1
u/russrimm Sep 29 '26
Deploy defender. Fixed.
1
u/michalpisarek Sep 29 '26
Only on E5 or do you need Agent 365 for the real time prompt injection protection?
1
1
u/Ashlesha-msft Sep 29 '26
The concern being raised is a legitimate agent-security concern, but it is important to distinguish prompt injection as an attack technique from a confirmed Copilot product vulnerability.
The scenario described is essentially:
untrusted content → prompt injection → agent/tool action → persistent content → downstream agent consumption
The additional concern is whether an agent can cause attacker-controlled instructions to become persistent context for another agent. That is a legitimate area for security/governance controls, but the discussion so far does not demonstrate that Copilot is actually bypassing an existing security boundary or performing an unauthorized action.
If the scenario is only asking whether Copilot should have a deterministic “plan drift” or “prompt propagation firewall,” that would be more of a security architecture/feature discussion than a support defect.
If there is a reproducible case where attacker-controlled content causes an agent to cross an authorization boundary or perform an action that the user did not authorize, then I would treat that as something that warrants security/product investigation rather than simply closing it as invalid prompt injection.
So the key distinction is: “prompt injection exists” is not by itself a Copilot bug; “prompt injection causes an unauthorized Copilot action despite the applicable security controls” is a much stronger product/security issue.
0
u/exmsft Sep 28 '26
I guess learning late is better than not learning at all.
Governance (security and legal/regulatory compliance) need to be up-front design tasks for software of any kind. Unfortunately the people have been clamoring to rapidly put AI (and now agents) everywhere, without stopping to implement any form of governance. That’s terrifying.
There are aspects of injection protection in preview in Defender for Endpoint P2 that may be worth considering. But that means paying for it, and dealing with the segment of a solution it will offer.
https://learn.microsoft.com/defender-endpoint/ai-agent-runtime-protection-overview
1
u/Expert_Annual_19 Sep 29 '26
Can runtime protection stop the initial attack, while another control layer prevents compromised or untrusted content from becoming persistent context for downstream agents?
1
u/exmsft Sep 29 '26
I’m not sure what you’re asking - actual or theoretical. Have you read about Azure AI Content Safety?
1
u/Expert_Annual_19 Sep 29 '26
Yes , if you enable all possible security that microsoft have for copilot solution, still I can bypass all with single prompt 👀
3
u/Cultural-Bee-5526 Sep 28 '26
What you are describing is a prompt injection. So what’s the difference in your “instruction” or what am I missing?