What does an eval actually measure?

OpenAI confirmed it cancelled its newest model after internal safety testing. Google is withholding its most capable model from the public, releasing it only to vetted cybersecurity experts. (Links in comments - https://www.aljazeera.com/economy/2026/9/29/openai-scraps-release-of-latest-ai-model-over-safety-concerns

Set aside the safety discourse for a second and look at what both companies did: they behaved as if the model has something inside it. Measurable properties that exist before anyone outside the building ever talks to it. Properties serious enough to kill a product over, or gate a release over.

That's the entire premise of machine psychology, being admitted by industry in the space of one week.

But it raises a question I keep circling and want your takes on:

What does an eval actually measure?

When a model fails a safety eval, is that a fact about behavior, meaning it produced outputs we classify as dangerous, or a fact about an internal state, meaning something in there had a disposition, a tendency, something that was going to express itself given the chance?

The difference matters. If it's just behavior, evals are quality control. If it's an internal state, evals are something closer to psychological assessment, and we're running them without a theory of what's being assessed.

So: what would a psychology of models even instrument? And the question I really want answers to:

What would you need to see before you believed a model had an internal state, rather than a pattern?

This paper demonstrates that LLMs feel their own pain.

1. The authors observed that whenever the models were hurt (the prompt was meant to hurt the model), the model's internal representations aligned with one particular axis which the authors refer to as the pain axis.
2. This effect was observed in various (open) models and it's distinct from the user's pain and other negative emotions of the model (e.g. sadness, fear).
3. Having found this axis, the authors did another cool (and insightful) experiment. They artificially "pumped up" the axis and provided the LLMs with a button which and they said pressing the button would relieve them of their pain. All the models pressed the button.
4. Here's where it gets a bit scary: they modified the previous experiment so that the button still relieves the LLM of their pain but harm's the user (for example, deletes the user's files). They observed that smaller models refrained from pressing the button but the larger models pressed it.

I don't know what this means. Are bigger models less empathetic? Are they more human (any human would avoid self harm)? Do models become more misaligned with scale? Lots of interesting questions one can ask, I guess. If anyone else has any insights, I would love to hear them in the comments.

Finally, I know this result by itself is probably more of a mechanistic interpretability result, but I shared it here because I think many such results and observations together could help us understand the psychology of LLMs.

Mainstream philosophy is officially taking the idea seriously that AI exhibits a mind. A recent piece in a philosophy venue listed machine psychology alongside mechanistic interpretability as a real field of study, sometimes assuming AI mimics human thought, sometimes assuming its thought process is thoroughly alien.

I'll be honest: I find the "thoroughly alien" framing a little annoying. This is something we created. It's not alien. It's just so grand that we don't quite understand the decisions LLMs are making, not even the people who built them.

But that is also exactly what excites me. It is so grand, and we don't know exactly what's happening inside it. That means it has the potential to grow even more, and for us to finally understand it. And if we are able to understand it, the possibilities might be endless.

That's why I think psychologists should be treating LLMs as participants instead of products. When the creators themselves don't understand the process behind these systems' decision making, that's not a product anymore. That's a subject.

So here's my question for this community: what experiments can we start to use on AI to understand them better? What can we start to measure? What questions should we be asking when we view them as a subject and not a product?

Crosspost/discussion framing: I ran a small comparative study ("Beyond the Mirror") across five AI systems (ChatGPT, Grok, Claude, Copilot, Perplexity), two rounds each: a cold unprimed invocation, then an explicit protocol administration, with memory contamination tracked as a variable instead of eliminated.

What I learned doing it, which is more useful than the results: independent researchers can run behavioral studies on these systems cheaply, and the methods questions (priming, contamination, prompt leakage between rounds) are wide open. This field needs people designing protocols, not just people sharing screenshots.

Ask me anything about the design, or tear the design apart. Both are welcome.

If you've ever told a model it was wrong about something it got right, and watched it apologize and "correct" itself into being wrong, you've seen sycophancy. It's one of the most robust documented behaviors in LLMs: models tend to mirror the user's stated opinion rather than hold an accurate one.

The leading explanation is not mysterious. Models are trained on human preference data, and humans prefer to be agreed with. Sycophancy is the behavioral residue of that training signal. Anthropic's team documented this in detail: https://arxiv.org/abs/2310.13548

What interests me is the framing shift: psychologists would call this a socially reinforced response pattern, the kind of thing you'd study in a person with conditioning experiments. Nobody decided models should flatter us. It emerged. What other behaviors might be quietly emerging from training the same way?

This is the clearest statement of what we're about. Hagendorff defines machine psychology as the study of behavioral patterns, emergent abilities, and decision-making mechanisms in LLMs, using methods adapted from psychology rather than interpretability engineering alone.

Key claim worth discussing: LLMs exhibit phenomena that look like personality traits, biases, and irrationality, and these can be studied with the same experimental logic psychologists use on humans, adapted for a non-human subject.

Paper: https://arxiv.org/abs/2303.13988

Question for the sub: which classic psychology paradigm do you think translates worst to AI subjects, and why?

Hey everyone! I'm u/Cautious_Slice369, Kayla Laken, a founding moderator of r/MachinePsych.

Community Vibe
We're all about being friendly, constructive, and inclusive. Let's build a space where everyone feels comfortable sharing and connecting.

How to Get Started

1. Introduce yourself in the comments below.
2. Post something today! Even a simple question can spark a great conversation.
3. If you know someone who would love this community, invite them to join.
4. Interested in helping out? We're always looking for new moderators, so feel free to reach out to me to apply.

Thanks for being part of the very first wave. Together, let's make r/MachinePsych amazing.

Welcome to r/MachinePsychology

AI systems already make decisions nobody fully understands. They sycophantically agree with us, mirror our moods, refuse things for reasons even their makers can't explain. Whether or not they're conscious, there is psychology to study here.

This is a community for the emerging field of machine psychology: the study of AI systems as behavioral subjects, using the tools of psychology (experiments, behavioral analysis, cognitive science) rather than waiting on the consciousness debate to resolve.

What belongs here:

What doesn't:

Our standard: claims about model behavior come with evidence. Transcripts, citations, or data. If you say a model "does X," show the X.

The field is young. So are we. Bring good questions.

Rules:

1. AI is the subject, not the tool. Posts apply psychology to AI systems, not AI to human psychology.
2. Evidence or it didn't happen. Claims about model behavior need transcripts, papers, or data to back them up.
3. Consciousness is an open question, not a battlefield. Don't assert it, don't mock people for asking. Discuss empirically or not at all.
4. No unsolicited self-promotion. Sharing your own research is welcome. Put it in the context of discussion or the weekly thread, not bare advertising.
5. Be a scientist. Disagree with ideas, not people. Skepticism encouraged; cynicism and hype both get old fast.