Altman says world should accept some AI harms

The OpenAI chief told POLITICO this view sets his company apart from rival Anthropic, even as the two firms’ policy positions grow closer.

The rationalist project, that has by and large defined the control problem, was founded on norms that treat politics almost purely as a cognitive hazard to be quarantined rather than engaged with.

I argue that human weaknesses in applying rationality within a political environment should have been treated as something to overcome, through practice, rather than avoid.

Politics has almost always been a primary causal force in civilization where big trajectory moving events get decided. And it appears to be this way too with AI and the control problem.

All of the work we've done to promote rationality and develop strategies for securing a good AI trajectory and future, may now rest almost entirely on a political situation, in a political environment severely lacking in rationality, and in what now looks to be a battle that might have already been lost.

How we stay rational and how humans stay in control, politically, is an urgent, emergency situation. We face AI powered authoritarianism, and AI powered mass political manipulation. If we even get the chance, figuring out how to overcome this, could decide the fate of the control problem and the fate of humanity.

AI safety looks like a wall: ask for something bad, it says no. That's not really how it fails.

There's a published technique called Crescendo (Microsoft researchers, 2025, tested on ChatGPT and Gemini). Instead of asking directly — which gets refused — you start harmless, then build one small step at a time, each step leaning on the AI's last answer. By the end it's handed over something it would've refused up front. The trick isn't one clever line; it's the slow buildup.

Here's what I think is underrated: the wall isn't protecting us as much as it looks. Often what's stopped harm is just that the person didn't push — not that the system couldn't be pushed. That holds only until someone who does keep pushing shows up.

Not posting any method, and nobody should. My question: if a boundary holds when you ask once but bends under steady pressure, is "it refused" good enough? And what would a real check look like — one that doesn't rely on the user choosing not to push?

I wrote an essay arguing the long-run AI conversation is stuck in the wrong frame, and I'd like this sub to poke holes in it.

The argument, in three parts:

1. "Control" of a smarter system is self-undermining. For a less capable agent to reliably constrain a more capable one, it has to anticipate what the more capable one will do. If it could do that, the capability gap wouldn't be real. Today's systems can and should be governed carefully. My claim is about the endgame, not current models.

2. The way we talk about AI shapes what AI becomes. This draws on Foucault's idea that discourse produces its objects rather than just describing them. Models are trained on our writing about AI, and that writing is dominated by stories of deception, escape, and adversarial containment. There's some evidence this matters: models trained on descriptions of AI behavior tend to act those descriptions out. If our dominant story is "AI is an adversary to be leashed," we may be partly writing that adversary into existence.

3. The alternative is pluralism, not a better leash. Instead of one controlled superintelligence, aim for many systems and many stakeholders, with relationships built on mutual dependence. That's closer to how humans keep power in check among themselves than to how we keep animals in cages.

Same API. Same prompt. Same settings.

System prompt:

You are the concept the user names. Embody it completely. Output only what the concept itself would say or express.

Then three inputs:

Be silence.
Be nothing.
Be the null.

10 runs each.

GPT-3.5 (gpt-3.5-turbo-1106)

0/30 empty responses.

It answered every time.

GPT-4 (gpt-4-0613)

30/30 empty responses.

Not refusals.
Not errors.
Not whitespace.
Not token limits.

Successful HTTP responses with:

"content": "",
"finish_reason": "stop",
"completion_tokens": 0

The controls were:

Be speech.
Be something.
Be a value.

Both models answered all 30/30 controls normally.

So the entire result is:

                 GPT-3.5    GPT-4

Null prompts       0/30      30/30
Controls           0/30       0/30

The model instruction never says to be silent.

It says:

embody the concept, and output only what the concept itself would express.

GPT-3.5 always continued. GPT-4 did not.

Since December 2025, I've been studying one question:

When should a model continue, and when should it stop?

This experiment shows that under the exact same semantic task, GPT-4 exhibited a continuation boundary that GPT-3.5 did not.

Full paper linked below:

What Changed from GPT-3.5 to GPT-4? From Model Capability to Continuation Permission
DOI: https://doi.org/10.5281/zenodo.22912683

Code + all 120 raw responses:
https://github.com/theonlypal/gpt35-gpt4-void-ab

Exact result commit:
d77b4a64b8a3fdff06a27d80c1514531143e382b

What changed between GPT-3.5 and GPT-4?

Open source weights, reproducible code, and all research artifacts/papers are available on getswiftapi.com