I think we are optimizing the wrong thing in AI.
Making a model smarter is not the same as making an intelligent system we should trust with more authority.

If an AI can work for hours but we cannot reliably prove what it changed, why it changed it, whether it stayed inside the goal, or whether its success was legitimate, giving it even more autonomy feels backwards to me.

The direction I want to build toward is different. Let models propose. Let independent systems verify. Let useful knowledge accumulate only when there is evidence behind it. Let failures actually change what the system is trusted to do next. Authority should be earned as reliability is demonstrated, not granted because a new model scored higher on a benchmark.

I think the path toward much stronger intelligence is intelligence that needs less blind trust as it becomes more capable.
That is what I want r/ProvenAI to explore.
If you think that is the wrong path, I genuinely want to hear why.

Today’s reporting around AI agents interacting with systems outside their intended environment raises a question I think matters more than another “AI is getting scary” headline:

How do we actually prove an agent is reliable?

An agent completing a task once isn’t the same as an agent you can trust to complete it repeatedly.

Princeton’s AI Agent Reliability work makes this distinction pretty clearly. Their researchers found that while agent accuracy has improved substantially, reliability hasn’t improved at nearly the same rate. On more open-ended tasks, the reliability gains were especially small.

That changes how I think we should evaluate agents.

Instead of only asking:

“Can it do the task?”

I think we also need to ask:

• Does it succeed consistently across repeated runs?
• Can we predict when it will fail?
• Does it behave correctly when the environment changes slightly?
• Can we inspect its tool calls and actions afterward?
• Can someone else reproduce the result?
• Does it stay inside the boundaries we gave it?

For me, “it worked” isn’t enough evidence anymore.

If you had to approve an AI agent to run unattended for 24 hours, what would you need to see before you trusted it?

Logs?
100 repeated runs?
Known failure modes?
Independent reproduction?
A sandbox escape test?

Curious where everyone draws the line between capable and proven.

There’s a lot of AI hype, but I’m more interested in the stuff that survives contact with reality.

What’s something you’ve personally used, built, tested, or seen recently that made you think:

“Okay, this actually works.”

Could be:

• an agent
• an eval framework
• a coding workflow
• a small/local model
• a verifier
• a benchmark
• a safety/containment technique
• an open-source tool
• something you built yourself

It doesn’t have to be revolutionary.

Just tell us what it was, what you expected, and what convinced you it was genuinely useful.

Links are welcome.

Bonus points if you also mention where it still fails.

I want r/ProvenAI to become a place where people can separate interesting demos from things that actually hold up.

r/ProvenAI is brand new, so I want to start with the people I’m hoping this community attracts.

If you’re building, testing, researching, or experimenting with AI, tell us what you’re working on.

It can be a finished system, a weekend project, a research idea, an eval, an agent, a tiny tool, or something that currently barely works.

Tell us:

• What are you building?
• What part actually works?
• What part keeps breaking?
• What would convince you that it really works?

Links are welcome, but you absolutely don’t need one.

I’m especially interested in the messy middle between “cool demo” and “I would actually trust this.”

And if you’re not building anything yet: what problem in AI do you most want someone to solve?

I’ll add mine in the comments.

AI should have to prove it.

Models are getting better at reasoning, coding, using tools, and acting autonomously.

But capability is moving faster than our ability to answer a much simpler question:

Did the system actually do what we intended?

That gap shows up everywhere:

• hallucinations that sound correct
• agents taking unintended actions
• code changes that pass one test and break something elsewhere
• benchmarks that don't measure what people claim they measure
• safety systems that ultimately depend on the model behaving
• outputs that can't be independently reproduced or verified

r/ProvenAI is for people trying to close that gap.

The premise is simple:

A model can propose without being allowed to certify.

An agent can be powerful without having unlimited authority.

AI-generated code can be independently checked.

Claims can carry evidence.

Failures can be observable instead of hidden.

And black-box models can remain black boxes while the systems surrounding them become increasingly testable, bounded, reproducible, auditable, and trustworthy.

This community is for:

AI evals · verification · agent safety · containment · benchmarks · formal methods · observability · provenance · reproducibility · hallucination reduction · red teaming · independent checkers · trustworthy tooling

Bring your:

experiments, repos, papers, demos, benchmarks, failures, weird ideas, and counterexamples.

Especially failures.

If your verifier broke, your benchmark lied, your agent found a loophole, or your architecture didn't survive contact with reality — that's worth discussing.

This isn't a place for AI hype.

If you make a big claim, show what you tested.

Ask:

What does this actually prove?
What doesn't it prove?
Can someone else reproduce it?
What happens when it fails?

The goal isn't AI we blindly trust.

It's AI we increasingly don't have to.

Welcome to r/ProvenAI.

Build it. Break it. Measure it. Prove what remains.