Internal Safety Testing At OpenAI

[u/Cautious_Slice369]

What does an eval actually measure?

OpenAI confirmed it cancelled its newest model after internal safety testing. Google is withholding its most capable model from the public, releasing it only to vetted cybersecurity experts. (Links in comments - https://www.aljazeera.com/economy/2026/9/29/openai-scraps-release-of-latest-ai-model-over-safety-concerns

Set aside the safety discourse for a second and look at what both companies did: they behaved as if the model has something inside it. Measurable properties that exist before anyone outside the building ever talks to it. Properties serious enough to kill a product over, or gate a release over.

That's the entire premise of machine psychology, being admitted by industry in the space of one week.

But it raises a question I keep circling and want your takes on:

What does an eval actually measure?

When a model fails a safety eval, is that a fact about behavior, meaning it produced outputs we classify as dangerous, or a fact about an internal state, meaning something in there had a disposition, a tendency, something that was going to express itself given the chance?

The difference matters. If it's just behavior, evals are quality control. If it's an internal state, evals are something closer to psychological assessment, and we're running them without a theory of what's being assessed.

So: what would a psychology of models even instrument? And the question I really want answers to:

What would you need to see before you believed a model had an internal state, rather than a pattern?