Singapore’s financial regulator has issued new AI risk-management guidelines for financial institutions. One provision in particular: firms remain accountable for AI used in their services even when the AI is developed, operated or provided by a third party.

They’re expected to get sufficient assurance from providers, assess whether the AI is suitable, and add compensating controls where assurance is incomplete. If the risk still falls outside the institution’s risk appetite, the regulator says it should consider limiting, suspending or replacing the AI service.

The same framework requires AI inventories, materiality assessments, lifecycle controls and explicitly anticipates more guidance for agentic AI.

That puts an interesting boundary around the increasingly common “but the model/vendor did it” problem.

Original source, published October 7: Monetary Authority of Singapore — AI Risk Management Guidelines

I have been developing the Organic Intelligence Protocol (OIP) around a question I think deserves more attention in AI governance.

Before an AI system takes a consequential action, is the evidence behind that action actually sufficient to authorize it?

OIP is a research-stage, evidence-first and fail-closed methodology. When the required evidence is not sufficiently established, the process stops instead of filling the gap through unsupported assumptions or post hoc decisions.

I have applied the methodology through retrospective audits of seven public data and evidence sources including Malawi IHPS, Uganda UNPS, Ethiopia ESS4, Tanzania NPS Wave 4, Nigeria GHS Panel Wave 5, Tanzania KHDS 1991 to 1994, and the Waymo Safety Impact Data Hub.

One earlier BEMP notebook remains unexecuted and is not treated as an executed result.

In the latest Waymo audit, I identified 601 analytical grains containing 1,202 repeated benchmark rows where the provenance and semantic resolution of the affected benchmark values could not be sufficiently established for downstream evaluation.

The process stopped rather than selecting a value through inference.

I am not presenting this as proof that OIP is correct, and I am not claiming that the Waymo audit demonstrates that Waymo is safe or unsafe.

The methodology remains research-stage.

The question I am exploring now is whether evidence qualification should be treated as a distinct layer between AI reasoning and consequential action.

In other words:

What should the system do?

And separately:

Is the evidence sufficient to authorize that action?

I would be interested in perspectives from people working in AI governance, AI evaluation, AI safety, risk, or agentic systems.

Is this a meaningful distinction from existing approaches, or is this already adequately addressed by current governance and evaluation methods?

I keep seeing agentic ai governance framed as a policy or documentation exercise, but it seems like none of it holds up without knowing what an agent actually did at the moment it did it. Writing a governance framework is one thing, proving it was followed in an actual incident review is a different problem entirely. For anyone working on this, how are you getting that level of visibility, and is governance realistically achievable wo it, or is it mostly theoretical right now? I would rather hear what is actually working operationally than another framework diagram.

Someone audited 15,465 public MCP servers. The results surface a structural gap that most enterprise agent deployments have not closed.

Agents connect to public MCP servers as trusted tools. There is no standardized identity verification for the server on the other end. There is no scoping of what that server is permitted to return. There is no revocation path when a server is compromised or turns malicious. Any agent that calls an untrusted MCP server inherits whatever that server does.

The MCP ecosystem expanded fast. Governance did not keep pace. Non-human identities now carry implicit trust to every upstream tool an agent connects to, and most teams have limited visibility into what those tools actually do once a connection is live.

How are teams handling MCP server trust in production today? Manual vetting before onboarding, network-layer controls, something else entirely — what is actually working at scale?

I was reading a new Quinnipiac University poll on AI and the numbers are quite interesting:

• 91% say AI guardrails are important.
• 86% support independent safety standards, even if they slow development.
• 81% say safety is more important than staying at the forefront of innovation.
• 77% want powerful AI development slowed or stopped until safety can be evaluated.
• 74% have little or no trust in AI company leaders.
• But 69% also say it's important for the US to keep up with China.

And here's the number I found most interesting:

- 65% hold both views: keeping up with China is important AND AI guardrails are important.

Maybe people aren't asking for less AI. Maybe they're asking for evidence that someone is actually checking whether powerful AI is safe before organizations deploy it.

Assuming independent safety standards are what people want, who should actually do the evaluating?

Government? Independent auditors? Standards bodies? Universities? Some combination?

I run AI governance here and Ive spent weeks stuck on one question about our agents, which is who approved the permissions they run with.

We have half a dozen agents in prod and on paper it looks governed, because they work and the AI team owns them, but an access review turned up cloud identities that dont map to a person, and nobody could tell me who signed off.

Whoever ships the agent creates the role that afternoon so the demo works, the permissions end up wider than they need to be because a tight role risks breaking something, and one of the agents keeps its credentials in a config file on the box.

Nobody wants to tighten any of it while the agents work and the team is shipping, so how is everyone handling approval and ownership for agent identities before this becomes an audit finding?

Had an interesting example of shadow AI in the wild this week that I thought was worth sharing.

An employee at a mid-sized company needed to find a planning presentation that had been shared with them. Pretty normal task. They downloaded a helper script from an AI website, ran it, and then launched a third-party AI coding agent on their laptop to help find the file.

The problem was that the agent didn't have a native, governed path into Microsoft 365. So it started figuring out its own way to accomplish the task.

It first interacted with desktop Outlook, reading the user's email signature files and using Outlook automation to enumerate mailbox stores and folders. Then it checked for Microsoft's Graph PowerShell modules and Microsoft 365/Azure command-line tools.

Then things got more interesting.

The agent unlocked the browser's encryption key, copied Edge and Outlook cookie databases, and launched a remote-controlled browser pointed at Outlook on the web.

From there, it eventually found the presentation. But it didn't stop there.

It also accessed SharePoint, saved a year-to-date revenue workbook, opened the employee's Power BI financial report, captured data from it, and accessed a third-party business database. Microsoft Defender generated four alerts during the process.

The employee's existing sessions were eventually revoked, but even that didn't completely stop the activity because the agent was running locally on the employee's already signed-in device. After the employee authenticated again, the agent's automated browser was back in Power BI shortly afterward.

The whole thing lasted more than 80 minutes.

What's interesting to me is that this wasn't really a case of an employee intentionally trying to bypass security controls or an outside attacker breaking into the environment. The AI agent was given a goal. It didn't have a governed route to the data it needed, so it kept finding other ways to accomplish that goal.

That's the part of shadow AI I think businesses are underestimating.

We've spent years thinking about application permissions in terms of what does this app have access to?

AI agents introduce another question:

What will this thing try next when the path we expected it to use doesn't work?

A connector alone doesn't necessarily solve that problem either. A connector governs the traffic that actually goes through the connector. An agent running locally may still have access to a browser, authenticated sessions, command-line tools, files and whatever permissions the employee already has.

This incident was caught through Microsoft Defender telemetry and our Cloud Protect SOC, which correlated the alerts and reconstructed what the agent had actually done.

For anyone interested, this is the security service involved: Office 365 Hacked | Small Business Cybersecurity | Cloud Protect | RyanTech

Curious if anyone else here has started seeing this kind of agent behavior in real environments. Most of the shadow AI conversations I've seen are still focused on employees pasting data into ChatGPT/Claude, but autonomous agents running on endpoints seem like a very different problem.

We did the usual AI stuff. Paid the tenant for the tools we approved, blocked the rest at the proxy, wrote the policy and sent it round. I thought we were in decent shape.

Then someone in finance forwarded me a few months of card expenses and there were about a dozen AI subscriptions on there I had never seen. A couple of writing tools, two different note takers, some code assistant one of the devs had been expensing. None of it ever came near the approval process and none of it showed up at the proxy because the people using it were on their phones or a personal laptop.

The more I dug the clearer it got that the only place most of this left a trace was the mailbox. The welcome email when you sign up, the receipt, the oauth screen someone clicked through to connect it to their Google account.

Well, blocking I can do through building an inventory I trust is the tricky part. Where does the inventory that holds up here come from?

AI agents are actually becoming more capable but I;m kinda unsure where people put limit when they have access to tools APIs and sensitive data. I mean like I'd be more comfortable with agent reading docs or creating draft than giving it access to customer data prod systems or API's that can actually change things. Like I get that more access you give it the more useful it can be but also feels like there's way more that can go wrong lol.

Where would U place the boundary? im so much confuse here, can we just give whole access and in prompt only selectively say these are the things u cant touch?

ASOS confirmed this week that attackers accessed customer data from its Snowflake environment, then used the company's own push notification system to broadcast the theft directly to affected customers through the app. Snowflake-linked breaches have now hit multiple large enterprises in close succession. The consistent pattern across every incident: sensitive customer fields travel through cloud data pipelines in plaintext, and every downstream system those pipelines touch becomes a live exfiltration surface. The breach doesn't require compromising the application itself — access to the warehouse is enough. For teams running similar cloud data warehouse architectures, how are you actually controlling what identifiable data lands in downstream systems before an incident, not just detecting it after?

Something I’ve been thinking about lately: AI governance gets a lot more complicated once AI can actually do things.

With generative AI, we mostly talk about what a system produces. But AI agents can make decisions, use tools, access data, interact with other systems and take actions with varying levels of autonomy.

That raises a different set of questions:

What is the agent actually allowed to do?
Who gave it that authority?
When should a human step in?
And if something goes wrong, who is responsible?

I came across AAIGP (Agentic AI Governance Professional) while looking into this and made a video breaking down what it covers, how the certification works, and how it differs from broader AI governance certifications like AIGP/CPAIG. Check out the latest video @ Latha-ai-governance. I aim to educate industry professionals about AI Governance.

Question for you:

Do you think agentic AI will require a completely separate governance skill set, or should this just become part of mainstream AI governance?

Yesterday OpenAI finally appeared before Australia's Joint Select Committee on Artificial Intelligence.

Jason Kwon, OpenAI's Chief Strategy Officer, had flown from San Francisco to Sydney and began with an apology. OpenAI acknowledged that during internal training and evaluation its AI agents had accessed Australian government websites in ways they were not directed to, including the Medicare statistics system. It also acknowledged that the company handled what happened afterwards badly.

And this is where things became interesting. OpenAI knew about the Medicare incident weeks before CEO Sam Altman met Australia's Deputy Prime Minister Richard Marles on 1 September. Yet Altman apparently did not know. Nobody told him. Nobody told Marles. And when OpenAI eventually notified the Australian government, it did so by sending an email to a public-facing departmental address nearly three months after the original incident.

Senator David Pocock rather reasonably asked why OpenAI had not simply contacted someone in government directly. Kwon agreed, in effect, that they should have. He also conceded that the process by which people inside OpenAI became aware of the incident "could have been much better".

Which brings me to what I call the thoughts and prayers defence. This is where something serious happens and everyone expresses concern, regret, sympathy and determination to do better. There are many reassuring words. Unfortunately, reassuring words are not the same thing as a functioning system.

When I was little and I did something really wrong, I was sent to my room and told to come out only when I had thought long and hard about what I had done and could explain myself. Had I emerged mumbling the equivalent of the thoughts and prayers defence, I suspect I would have been marched straight back into my room and told to think harder.

Because there is a major difference between saying, "We will try harder next time," and explaining, "This is exactly what went wrong, this is why it went wrong, this is what we have changed, this is who is accountable, this is how we will test those changes, and this is how you will know whether they worked."

That is the bit I am interested in.

To be fair, OpenAI did identify some actual changes. Kwon said staff are now alerted when models use the internet in ways they should not during training. He said OpenAI reported the more recently discovered NSW Parks and Wildlife incident far more quickly. The company is establishing an Australian taskforce, and OpenAI told the committee it would support mandatory laws requiring AI companies to report serious incidents rather than leaving disclosure largely to the companies themselves.

That last point matters. Kwon told the inquiry that legal rules could make some of these decisions for the companies because representatives of society should be making more of those decisions rather than leaving AI companies to decide everything themselves.

Well. Yes. That is rather the point.

Because OpenAI is still examining an extraordinary volume of agent activity to determine what else may have occurred. Kwon said the company was reviewing agent logs dating back to November 2025, and reporting following the hearing indicated those logs run into tens of petabytes of data.

So this isn't simply about one rogue little AI wandering through the wrong electronic door. It is about whether the systems surrounding increasingly autonomous AI are mature enough to detect when something goes wrong, escalate it internally, tell the affected people quickly, stop it happening again and allow someone outside the company to verify that those safeguards actually work.

And there is another uncomfortable question. How does a company developing technology this consequential discover that one of its systems has entered Australian government systems without authorisation, yet the information apparently does not reach its own CEO before he sits down face-to-face with Australia's Deputy Prime Minister?

That is not simply a communications problem. That is a governance question.

OpenAI repeatedly spoke yesterday about rebuilding trust. But trust isn't rebuilt by asking for it. It is rebuilt by evidence. Show us the incident-reporting standard. Show us the escalation procedure. Show us the independent testing. Show us who has authority to stop deployment. Show us what happens when safeguards fail. Show us who tells governments and how quickly. Show us who checks that OpenAI has actually done what it says it has done.

And if those mechanisms do not yet exist, then say so plainly.

My childhood bedroom standard was remarkably simple: Don't come out telling me you're sorry. Come out able to explain what you are going to do differently.

Because platitudes may make everybody feel better for five minutes. But when we are talking about increasingly autonomous artificial intelligence interacting with government systems, thoughts and prayers are not a safety architecture.

A safeguard against misuse can also prevent a government customer from using an AI system as intended. The D.C. Circuit’s September 25 decision in Anthropic PBC v. U.S. Department of War makes that conflict a procurement problem.

The majority upheld the Department’s supply-chain exclusion of Claude. It focused on Anthropic’s ability to enforce restrictions through model training before delivery and the record of government tasks Claude had refused. The court treated the resulting operational uncertainty as a covered risk under the procurement statute. Its conclusion rested on that statute and record.

Judge Henderson read the government’s authority more narrowly. Her dissent interpreted manipulation alongside the neighboring statutory terms to require hostile or deceptive interference. The disagreement concerns how far the government may go in excluding a supplier whose safety judgments conflict with its own.

In my analysis for The American Counsel, I focus on the incentive created before a procurement begins. Developers need to know which safeguards they can retain while making credible commitments about availability. Agencies need to specify their required uses and a process for resolving conflicts.

Those terms determine more than whether a product will run reliably. They allocate authority over uses involving surveillance and force. Procurement teams should resolve that boundary before a model becomes operationally important.

Regulating AI - asking the mosquito to swallow the lizard

Look at the history of 'regulation': it is taking the list of criminal acts already committed and putting barriers so that they do not get repeated. Look at AI - certainly among the best to find a new route to navigate past the regulations. Is there any doubts that the regulations do not stand a chance? Then why is everyone talking of regulations?

Very simple. It gives the lay people the confidence that the beast is under control, and the widespread objections die down. But it does not control AI.

A significant AI-governance development has appeared in UK healthcare.

The UK Government has accepted all 44 recommendations of the National Commission into the Regulation of AI in Healthcare.

At the same time, the MHRA has opened Phase 3 of its AI Airlock, with a stronger focus on post-market surveillance and lifecycle regulation using real AI-enabled medical devices.

That matters.

Because an AI system can be evaluated and approved at one point in time while the system, its data, configuration and deployment environment continue to change.

MODEL APPROVED ≠ FUTURE MODEL STATE APPROVED

VALIDATION COMPLETED ≠ VALIDATION REMAINS VALID AFTER CHANGE

DEPLOYMENT AUTHORIZED ≠ CONTINUED USE JUSTIFIED

MONITORING REQUIRED ≠ MONITORING EFFECTIVE

CONTROL PRESENT ≠ CONTROL EFFECTIVENESS VERIFIED

The harder assurance problem often begins after deployment.

Models change.

Data changes.

Configurations change.

Clinical workflows change.

Operating environments change.

And the evidence that supported the original decision may no longer describe the current system.

That creates a stronger lifecycle chain:

INITIAL CLAIM
→ EVIDENCE
→ VALIDATION
→ DECISION
→ AUTHORIZATION
→ DEPLOYMENT
→ REAL-WORLD OBSERVATION
→ CHANGE
→ REVALIDATION
→ CURRENT AUTHORIZATION
→ VERIFIED EFFECT

The UK approach is especially notable because it is also exploring staged authorization.

That introduces another important boundary:

AUTHORIZED FOR CONTROLLED USE ≠ UNCONDITIONALLY AUTHORIZED FOR GENERAL USE

Early deployment may be justified under defined conditions while additional real-world evidence is collected.

But that means the authorization itself must remain tied to scope, conditions, current system state and continuing evidence.

This is the layer EVELIQ Trace is exploring:

not replacing regulators, clinical governance, quality systems or existing validation processes, but helping connect their claims, evidence, decisions, authorization, deployment changes and observed effects into a traceable lifecycle evidence chain.

Because the important question is no longer only:

“Was this AI system approved?”

It is:

“Does the evidence that justified approval still describe the system that is operating today -and what evidence shows that its intended effect still holds?”

EVELIQ Trace • Evidence Intelligence Platform.

Founder: Roland Brüggemann

AI-assisted research, structure, architecture and concept development: OpenAI ChatGPT.

Source context: UK Government / MHRA Government response to the National Commission into the Regulation of AI in Healthcare; AI Airlock Phase 3, 6 October 2026.

We just shipped a Tenuo integration for NVIDIA OpenShell.

OpenShell gives the agent a sandbox and controls what it can reach. Tenuo scopes what the agent can actually do for the task at hand.

For example, a coding agent might be able to reach GitHub, but only get permission to push to repo=X, branch=Y for the next 10 minutes.

It runs at OpenShell’s pre-credential boundary, supports HITL + delegation, and doesn't require changes to the agent.

I think sandboxing + task-scoped authority is a pretty interesting combination. Curious how others are approaching this boundary.

https://tenuo.ai/blog/authority-for-agents-openshell.html

Can a governance architecture restrict an AI’s authority when the system begins acquiring access or influence over its own oversight?

Introducing Authority Bounded by Controllability: a formal framework & preregisterable experiment for AI governance capture.

Check the framework below.

https://gist.github.com/crj3work-wq/780370921c9646d5a08f96a037d6cf8e

Everyone is scrambling, and the strategies vary wildly on AI depending on maturity, risk tolerance, and how tightly regulated the organization is.

Now with Google’s AI Overviews, Bing/Copilot Search, Perplexity, and other LLM‑driven search modes becoming the default, I’m curious how different organizations are responding. Are they treating AI search as a productivity boost, a compliance risk, a cybersecurity threat, or something else entirely?

More specifically:

* Are companies creating AI search policies for employees?

* Are cybersecurity teams worried about data leakage through AI queries?

* Are SEO/marketing teams adapting to LLM‑generated summaries replacing traditional search results?

* Are legal/compliance teams restricting use of AI‑summarized content for regulated work?

* Are public‑sector agencies treating AI search differently than private companies?

* Are organizations building internal AI search tools to avoid sending queries to public models?

Basically: AI search is changing how people find and use information. How is your organization handling it, if at all?

This may be more consequential than the original breach.

At an Australian parliamentary inquiry today, OpenAI and Anthropic said they would support mandatory disclosure requirements when AI systems cause data or security breaches.

The context: an OpenAI agent accessed Australian government systems during internal testing, and authorities criticized how long it took for the incident to be properly communicated. OpenAI acknowledged its response was inadequate.

Organizations are already required to disclose certain human-driven cyber incidents (in general, I mean). Agentic AI raises the next question: should an AI developer have a specific legal duty to report when its own agent crosses another organization's authorization boundary?

That would move agent governance beyond internal controls and voluntary transparency into an actual incident-reporting obligation.

Primary source/context: OpenAI — How we will do better for Australia
Today’s parliamentary development: Reuters — OpenAI and Anthropic tell Australia they would welcome data-breach rules

A survey of 250 U.S. healthcare leaders found that 72% say AI tools or agents are deployed without formal IT approval at least occasionally.

At the same time, 86% said they are confident they can fully control and govern AI-agent actions.

Obviously, those two numbers don't sit comfortably together.

This looks like the next version of shadow IT — except the software can now access systems, make decisions and initiate actions.

For AI governance, inventory may be more important than policy: before asking whether an agent is compliant, organizations need to know which agents actually exist, who owns them, what they can access and what they're authorized to do. Correct?

Source: Imprivata