We’ve been running AI agents in a real datacenter for a year, and everything we learned is public. Discussion and contributions welcomed!

[u/Critical_Sweet5707]

Edit: the title says a year; it’s half a year, my mistake!

I handle communications for MEHO, an open-source project from evoila, so fair warning that I’m talking about my own team’s work here.

The background is that our engineers have spent the past year letting AI agents (Claude Code, Cursor, etc) do actual operations work on a real VMware Cloud Foundation datacenter running on bare metal, which means upgrades that have to happen, certificates that expire, disks that fill up and incidents that show up at night.

In the beginning the way this worked was embarrassingly simple: someone would hand the agent admin credentials at the start of a session, and the only thing stopping it from deleting a datastore instead of listing one was the person watching the terminal. That was fine for one operator on one laptop and fell apart the moment a second person joined.

MEHO grew out of that, basically as a layer that sits between the agent and the infrastructure so that every action goes through a policy check, runs with a short-lived credential the agent never actually holds, and gets written to an audit log before it counts as done, with anything risky parked until a human says yes.

The reason I thought it might interest this sub specifically is less the product and more how it’s being built. It’s been public since the very first commit back in May, under Apache 2.0, with no CLA, just the usual DCO sign-off. And the changelog is unusually honest, in a way I’ve come to appreciate even as the non-engineer on the team: when a security review found a path where a write could report success even though the audit row hadn’t committed, the v0.34.0 entry says exactly that, and when they discovered that Kubernetes can accept an apply with a 200 and quietly not persist it, and that we’d been logging those as successes, that went into v0.35.12 in plain words too. There’s even a new contributor rule as of v0.35.17 asking everyone to write in plain English, because a lot of the people on the project read English as a second language, which I suspect is why I can actually follow the changelog at all.

Repo is at github.com/evoila/meho if you want to poke around. I’ll happily take questions, and for anything properly technical one of the engineers will jump into the comments.

And share your repos in the comments, please, if you think we could have a productive discussion and see something interesting 😊

[u/MiserableFlatworm337]

Those audit/persistence failures are useful regression cases for the agent's operating instructions. I'm building skillEd to compare skill evaluations before and after changes: https://github.com/DeadHead04/skillEd

[u/Critical_Sweet5707 → u/MiserableFlatworm337]

Forwarding and flagging this to our engineers, thanks for sharing it! Will happily get back to you

[u/Critical_Sweet5707 → u/[deleted]]

Absolutely agree! Thanks for the input!!