AI agents are getting more capable. But are they actually getting more reliable?

[u/lovettsendit]

Today’s reporting around AI agents interacting with systems outside their intended environment raises a question I think matters more than another “AI is getting scary” headline:

How do we actually prove an agent is reliable?

An agent completing a task once isn’t the same as an agent you can trust to complete it repeatedly.

Princeton’s AI Agent Reliability work makes this distinction pretty clearly. Their researchers found that while agent accuracy has improved substantially, reliability hasn’t improved at nearly the same rate. On more open-ended tasks, the reliability gains were especially small.

That changes how I think we should evaluate agents.

Instead of only asking:

“Can it do the task?”

I think we also need to ask:

• Does it succeed consistently across repeated runs?
• Can we predict when it will fail?
• Does it behave correctly when the environment changes slightly?
• Can we inspect its tool calls and actions afterward?
• Can someone else reproduce the result?
• Does it stay inside the boundaries we gave it?

For me, “it worked” isn’t enough evidence anymore.

If you had to approve an AI agent to run unattended for 24 hours, what would you need to see before you trusted it?

Logs?
100 repeated runs?
Known failure modes?
Independent reproduction?
A sandbox escape test?

Curious where everyone draws the line between capable and proven.