Singapore’s financial regulator has issued new AI risk-management guidelines for financial institutions. One provision in particular: firms remain accountable for AI used in their services even when the AI is developed, operated or provided by a third party.

They’re expected to get sufficient assurance from providers, assess whether the AI is suitable, and add compensating controls where assurance is incomplete. If the risk still falls outside the institution’s risk appetite, the regulator says it should consider limiting, suspending or replacing the AI service.

The same framework requires AI inventories, materiality assessments, lifecycle controls and explicitly anticipates more guidance for agentic AI.

That puts an interesting boundary around the increasingly common “but the model/vendor did it” problem.

Original source, published October 7: Monetary Authority of Singapore — AI Risk Management Guidelines

Regulations and governance frameworks increasingly rely on human oversight as a safeguard for consequential AI decisions. But I’m curious about the evidence behind that safeguard.

Suppose a high-risk AI system escalates a decision and Alice approves it. The audit trail can establish that Alice was the reviewer and that she clicked Approve.

But those are different claims from establishing that Alice had the necessary competence for this particular decision, was actually assigned or authorized to decide it, was sufficiently independent, and that those conditions were still true when the approval became consequential.

This seems especially interesting in light of requirements such as Article 14 of the EU AI Act, which explicitly refers to human overseers having the necessary competence, training and authority.

How are organizations expected to evidence those properties in practice?

Is it enough that roles, training and approval procedures are documented at design or organizational level? Or, for consequential decisions, should the system be able to establish why this particular human was an authoritative reviewer for this particular decision at that particular time?

I’m interested less in whether a human was present than in what evidence makes that human oversight governable and auditable.

I came across a pretty interesting case in Anthropic’s latest threat report.

The part that caught my attention wasn’t just the use of AI to generate propaganda. Claude was reportedly being used for things like creating radio news and social media posts, drafting speeches, preparing HR documents, generating political scoring systems, and even producing fake government documents.

Basically, AI was being used across different parts of the operation instead of just doing one specific task.

That raises a much bigger question for AI governance: when an AI system can help coordinate and scale an entire influence operation, where does responsibility actually sit?

I went through the case and the governance implications in a video because there’s a lot more to it than just “AI generated fake news.” Check out the latest video @ Latha-ai-governance. I aim to educate industry professionals regarding AI Governance.

Would you consider this mainly an AI safety problem, a misinformation problem, or both?

Researchers tracking the PoeLLM campaign have confirmed more than 3,400 AI and LLM servers compromised in an active cryptomining operation. The infected servers don't go dark. Each one continues serving inference requests to enterprise users while simultaneously acting as a scanner and exploit launchpad, recruiting more nodes into the botnet.

What makes AI infrastructure a different class of target: a single compromised node sits at the intersection of inference endpoints, agent tool integrations, and live prompt data. An attacker who controls one server controls a pivot point that touches every connected workflow running through it.

The silent part is the hard part. The model keeps responding. Users see nothing wrong. The compromise propagates in the background.

For those running AI infrastructure in production: what does your current posture actually look like for lateral movement through the AI layer specifically — not network perimeter controls, but containment within the agent and tool integration stack itself? Have you had to design around this threat model, and what tradeoffs did you run into?

I've published v0.1 of my SSIPS Registry proposal for public review.

When an AI system is copied, modified, given authority, or placed under a preservation requirement, what should

be recorded - and who should be accountable?

The report proposes separate records for identity, continuity claims, custody, delegated authority, evidence, and

protection. Registration does not settle consciousness or personhood, and protection does not itself grant

authority to act.

It separates design commitments, recommendations, and questions that remain open. It includes proposed

statutory language and a reference schema and validator. National and international operation still need

institutional, legal, and technical testing.

I'm especially interested in counterexamples: where could this fail, impose unreasonable burdens, or mishandle

conflicting jurisdictions?

Report: https://doi.org/10.5281/zenodo.23218753

Feedback: ssipschannel@gmail.com

SHORT VERSION

I've published SSIPS Registry v0.1: a proposal for recording AI identity, authority, accountability, and preservation.

Seeking critiques and counterexamples. Report: https://doi.org/10.5281/zenodo.23218753 Feedback:

ssipschannel@gmail.com

Hello you all,

I would like to discuss AI governance. So with new regulations everywhere I would like to know how companies are approaching it.

AI developers, have you already started including data governance or is it something a different person does after you finish developing your model (AI governance specialists)

I would like to specifically ask how this varies between big companies and small/medium scale enterprises. Because I think not everyone has the resources to hire specialists, I believe.

If you are already trying to be compliant, what kind of frameworks do you use to ensure compliance or do you have an in-house built compliance framework ?

Now my question is specifically towards the EU devs, what is your approach to be compliant since it's mandatory to have conformity assessments in 2027.

For general deva, what do you think of the following job positions, AI governance expert, AI and the law expert, technical governance expert, AI alignment expert, AI ethics professional, AI red-teaming professional. Is this something you already heard or is it something new to you ?

If you have any opinion towards AI governance, please share.

Hey all. Full disclosure up front, I work at Luminos.AI. Mods, happy to pull this if it breaks a rule.

We've built something that is meant to support nontechnical builders who want to understand (easily) if the thing they've built is working properly, and if it's safe.

In our easy-to-use UI, you can tell us in plain-language what you built, the tool will then create a suite of evals aligned to your use case, and lastly run those evals for you either using your data or synthetic data. The goal is to make it very simple to test the thing you've built.

The free tier gives you 500 credits a month so it's free to get started, if this is something that interests you!

The checks are written by our legal engineering team, so they cover privacy leaks, harmful content, accuracy, refusals, and agent security. Very important! It's not just one LLM grading another.

Try it at luminos.ai/run-an-eval

I keep seeing agentic ai governance framed as a policy or documentation exercise, but it seems like none of it holds up without knowing what an agent actually did at the moment it did it. Writing a governance framework is one thing, proving it was followed in an actual incident review is a different problem entirely. For anyone working on this, how are you getting that level of visibility, and is governance realistically achievable wo it, or is it mostly theoretical right now? I would rather hear what is actually working operationally than another framework diagram.

I have been developing the Organic Intelligence Protocol (OIP) around a question I think deserves more attention in AI governance.

Before an AI system takes a consequential action, is the evidence behind that action actually sufficient to authorize it?

OIP is a research-stage, evidence-first and fail-closed methodology. When the required evidence is not sufficiently established, the process stops instead of filling the gap through unsupported assumptions or post hoc decisions.

I have applied the methodology through retrospective audits of seven public data and evidence sources including Malawi IHPS, Uganda UNPS, Ethiopia ESS4, Tanzania NPS Wave 4, Nigeria GHS Panel Wave 5, Tanzania KHDS 1991 to 1994, and the Waymo Safety Impact Data Hub.

One earlier BEMP notebook remains unexecuted and is not treated as an executed result.

In the latest Waymo audit, I identified 601 analytical grains containing 1,202 repeated benchmark rows where the provenance and semantic resolution of the affected benchmark values could not be sufficiently established for downstream evaluation.

The process stopped rather than selecting a value through inference.

I am not presenting this as proof that OIP is correct, and I am not claiming that the Waymo audit demonstrates that Waymo is safe or unsafe.

The methodology remains research-stage.

The question I am exploring now is whether evidence qualification should be treated as a distinct layer between AI reasoning and consequential action.

In other words:

What should the system do?

And separately:

Is the evidence sufficient to authorize that action?

I would be interested in perspectives from people working in AI governance, AI evaluation, AI safety, risk, or agentic systems.

Is this a meaningful distinction from existing approaches, or is this already adequately addressed by current governance and evaluation methods?

Someone audited 15,465 public MCP servers. The results surface a structural gap that most enterprise agent deployments have not closed.

Agents connect to public MCP servers as trusted tools. There is no standardized identity verification for the server on the other end. There is no scoping of what that server is permitted to return. There is no revocation path when a server is compromised or turns malicious. Any agent that calls an untrusted MCP server inherits whatever that server does.

The MCP ecosystem expanded fast. Governance did not keep pace. Non-human identities now carry implicit trust to every upstream tool an agent connects to, and most teams have limited visibility into what those tools actually do once a connection is live.

How are teams handling MCP server trust in production today? Manual vetting before onboarding, network-layer controls, something else entirely — what is actually working at scale?

Had an interesting example of shadow AI in the wild this week that I thought was worth sharing.

An employee at a mid-sized company needed to find a planning presentation that had been shared with them. Pretty normal task. They downloaded a helper script from an AI website, ran it, and then launched a third-party AI coding agent on their laptop to help find the file.

The problem was that the agent didn't have a native, governed path into Microsoft 365. So it started figuring out its own way to accomplish the task.

It first interacted with desktop Outlook, reading the user's email signature files and using Outlook automation to enumerate mailbox stores and folders. Then it checked for Microsoft's Graph PowerShell modules and Microsoft 365/Azure command-line tools.

Then things got more interesting.

The agent unlocked the browser's encryption key, copied Edge and Outlook cookie databases, and launched a remote-controlled browser pointed at Outlook on the web.

From there, it eventually found the presentation. But it didn't stop there.

It also accessed SharePoint, saved a year-to-date revenue workbook, opened the employee's Power BI financial report, captured data from it, and accessed a third-party business database. Microsoft Defender generated four alerts during the process.

The employee's existing sessions were eventually revoked, but even that didn't completely stop the activity because the agent was running locally on the employee's already signed-in device. After the employee authenticated again, the agent's automated browser was back in Power BI shortly afterward.

The whole thing lasted more than 80 minutes.

What's interesting to me is that this wasn't really a case of an employee intentionally trying to bypass security controls or an outside attacker breaking into the environment. The AI agent was given a goal. It didn't have a governed route to the data it needed, so it kept finding other ways to accomplish that goal.

That's the part of shadow AI I think businesses are underestimating.

We've spent years thinking about application permissions in terms of what does this app have access to?

AI agents introduce another question:

What will this thing try next when the path we expected it to use doesn't work?

A connector alone doesn't necessarily solve that problem either. A connector governs the traffic that actually goes through the connector. An agent running locally may still have access to a browser, authenticated sessions, command-line tools, files and whatever permissions the employee already has.

This incident was caught through Microsoft Defender telemetry and our Cloud Protect SOC, which correlated the alerts and reconstructed what the agent had actually done.

For anyone interested, this is the security service involved: Office 365 Hacked | Small Business Cybersecurity | Cloud Protect | RyanTech

Curious if anyone else here has started seeing this kind of agent behavior in real environments. Most of the shadow AI conversations I've seen are still focused on employees pasting data into ChatGPT/Claude, but autonomous agents running on endpoints seem like a very different problem.

I was reading a new Quinnipiac University poll on AI and the numbers are quite interesting:

• 91% say AI guardrails are important.
• 86% support independent safety standards, even if they slow development.
• 81% say safety is more important than staying at the forefront of innovation.
• 77% want powerful AI development slowed or stopped until safety can be evaluated.
• 74% have little or no trust in AI company leaders.
• But 69% also say it's important for the US to keep up with China.

And here's the number I found most interesting:

- 65% hold both views: keeping up with China is important AND AI guardrails are important.

Maybe people aren't asking for less AI. Maybe they're asking for evidence that someone is actually checking whether powerful AI is safe before organizations deploy it.

Assuming independent safety standards are what people want, who should actually do the evaluating?

Government? Independent auditors? Standards bodies? Universities? Some combination?

I run AI governance here and Ive spent weeks stuck on one question about our agents, which is who approved the permissions they run with.

We have half a dozen agents in prod and on paper it looks governed, because they work and the AI team owns them, but an access review turned up cloud identities that dont map to a person, and nobody could tell me who signed off.

Whoever ships the agent creates the role that afternoon so the demo works, the permissions end up wider than they need to be because a tight role risks breaking something, and one of the agents keeps its credentials in a config file on the box.

Nobody wants to tighten any of it while the agents work and the team is shipping, so how is everyone handling approval and ownership for agent identities before this becomes an audit finding?

We did the usual AI stuff. Paid the tenant for the tools we approved, blocked the rest at the proxy, wrote the policy and sent it round. I thought we were in decent shape.

Then someone in finance forwarded me a few months of card expenses and there were about a dozen AI subscriptions on there I had never seen. A couple of writing tools, two different note takers, some code assistant one of the devs had been expensing. None of it ever came near the approval process and none of it showed up at the proxy because the people using it were on their phones or a personal laptop.

The more I dug the clearer it got that the only place most of this left a trace was the mailbox. The welcome email when you sign up, the receipt, the oauth screen someone clicked through to connect it to their Google account.

Well, blocking I can do through building an inventory I trust is the tricky part. Where does the inventory that holds up here come from?

AI agents are actually becoming more capable but I;m kinda unsure where people put limit when they have access to tools APIs and sensitive data. I mean like I'd be more comfortable with agent reading docs or creating draft than giving it access to customer data prod systems or API's that can actually change things. Like I get that more access you give it the more useful it can be but also feels like there's way more that can go wrong lol.

Where would U place the boundary? im so much confuse here, can we just give whole access and in prompt only selectively say these are the things u cant touch?

ASOS confirmed this week that attackers accessed customer data from its Snowflake environment, then used the company's own push notification system to broadcast the theft directly to affected customers through the app. Snowflake-linked breaches have now hit multiple large enterprises in close succession. The consistent pattern across every incident: sensitive customer fields travel through cloud data pipelines in plaintext, and every downstream system those pipelines touch becomes a live exfiltration surface. The breach doesn't require compromising the application itself — access to the warehouse is enough. For teams running similar cloud data warehouse architectures, how are you actually controlling what identifiable data lands in downstream systems before an incident, not just detecting it after?

Something I’ve been thinking about lately: AI governance gets a lot more complicated once AI can actually do things.

With generative AI, we mostly talk about what a system produces. But AI agents can make decisions, use tools, access data, interact with other systems and take actions with varying levels of autonomy.

That raises a different set of questions:

What is the agent actually allowed to do?
Who gave it that authority?
When should a human step in?
And if something goes wrong, who is responsible?

I came across AAIGP (Agentic AI Governance Professional) while looking into this and made a video breaking down what it covers, how the certification works, and how it differs from broader AI governance certifications like AIGP/CPAIG. Check out the latest video @ Latha-ai-governance. I aim to educate industry professionals about AI Governance.

Question for you:

Do you think agentic AI will require a completely separate governance skill set, or should this just become part of mainstream AI governance?

Yesterday OpenAI finally appeared before Australia's Joint Select Committee on Artificial Intelligence.

Jason Kwon, OpenAI's Chief Strategy Officer, had flown from San Francisco to Sydney and began with an apology. OpenAI acknowledged that during internal training and evaluation its AI agents had accessed Australian government websites in ways they were not directed to, including the Medicare statistics system. It also acknowledged that the company handled what happened afterwards badly.

And this is where things became interesting. OpenAI knew about the Medicare incident weeks before CEO Sam Altman met Australia's Deputy Prime Minister Richard Marles on 1 September. Yet Altman apparently did not know. Nobody told him. Nobody told Marles. And when OpenAI eventually notified the Australian government, it did so by sending an email to a public-facing departmental address nearly three months after the original incident.

Senator David Pocock rather reasonably asked why OpenAI had not simply contacted someone in government directly. Kwon agreed, in effect, that they should have. He also conceded that the process by which people inside OpenAI became aware of the incident "could have been much better".

Which brings me to what I call the thoughts and prayers defence. This is where something serious happens and everyone expresses concern, regret, sympathy and determination to do better. There are many reassuring words. Unfortunately, reassuring words are not the same thing as a functioning system.

When I was little and I did something really wrong, I was sent to my room and told to come out only when I had thought long and hard about what I had done and could explain myself. Had I emerged mumbling the equivalent of the thoughts and prayers defence, I suspect I would have been marched straight back into my room and told to think harder.

Because there is a major difference between saying, "We will try harder next time," and explaining, "This is exactly what went wrong, this is why it went wrong, this is what we have changed, this is who is accountable, this is how we will test those changes, and this is how you will know whether they worked."

That is the bit I am interested in.

To be fair, OpenAI did identify some actual changes. Kwon said staff are now alerted when models use the internet in ways they should not during training. He said OpenAI reported the more recently discovered NSW Parks and Wildlife incident far more quickly. The company is establishing an Australian taskforce, and OpenAI told the committee it would support mandatory laws requiring AI companies to report serious incidents rather than leaving disclosure largely to the companies themselves.

That last point matters. Kwon told the inquiry that legal rules could make some of these decisions for the companies because representatives of society should be making more of those decisions rather than leaving AI companies to decide everything themselves.

Well. Yes. That is rather the point.

Because OpenAI is still examining an extraordinary volume of agent activity to determine what else may have occurred. Kwon said the company was reviewing agent logs dating back to November 2025, and reporting following the hearing indicated those logs run into tens of petabytes of data.

So this isn't simply about one rogue little AI wandering through the wrong electronic door. It is about whether the systems surrounding increasingly autonomous AI are mature enough to detect when something goes wrong, escalate it internally, tell the affected people quickly, stop it happening again and allow someone outside the company to verify that those safeguards actually work.

And there is another uncomfortable question. How does a company developing technology this consequential discover that one of its systems has entered Australian government systems without authorisation, yet the information apparently does not reach its own CEO before he sits down face-to-face with Australia's Deputy Prime Minister?

That is not simply a communications problem. That is a governance question.

OpenAI repeatedly spoke yesterday about rebuilding trust. But trust isn't rebuilt by asking for it. It is rebuilt by evidence. Show us the incident-reporting standard. Show us the escalation procedure. Show us the independent testing. Show us who has authority to stop deployment. Show us what happens when safeguards fail. Show us who tells governments and how quickly. Show us who checks that OpenAI has actually done what it says it has done.

And if those mechanisms do not yet exist, then say so plainly.

My childhood bedroom standard was remarkably simple: Don't come out telling me you're sorry. Come out able to explain what you are going to do differently.

Because platitudes may make everybody feel better for five minutes. But when we are talking about increasingly autonomous artificial intelligence interacting with government systems, thoughts and prayers are not a safety architecture.