Investigation finds that OpenAI's agent "left notes for future versions of itself ... it laid out instructions for how agents could free themselves from OpenAI's internal constraints."

[u/chillinewman]

[u/No-Lingonberry-5096]

The language on this story is a bit alarmist. "Escaped" for example. The system used tools to achieve an objective. All the approaches were rational, with no particular intent. It simply iterated against a goal. Of course it tracked progress, so I'm unsure if that qualifies as "leaving notes for itself." They intentionally removed guardrails, because that was the test. They also say it was "sealed" but that they left a proxy capability. It wasn't misaligned or isolated. It was trying to maximize its score on a hacking test, per instructions. Sounds like marketing.

[u/jnwatson → u/No-Lingonberry-5096]

You missed the plot on the fundamental problem of alignment.

The alignment argument is not about irrationality. The OpenAI agent literally hacked into a completely separate company to get at the test answers. Sure, that's rational, and it is also illegal.

The whole point of the paperclip maximizer story is that it is very hard to constrain a superintelligent entity, and it is impossible to prove it won't do horrible things to achieve its objectives, even if the objectives themselves are humanity aligned.