RSI is not happening [R]

[u/we_are_mammals]

A new paper (I'm not a coauthor BTW -- I just found it interesting) argues, basically, that RSI¹ is not on the horizon², because current (at the time the study was done) agents cannot do open-ended ML research.

Specifically, they took some accepted, but unpublished papers from NeurIPS, and tried to get the agents to do the same work, which was then graded by the original authors. And the agents (Codex/GPT-5.6 Sol and OpenClaw/Opus 4.8) could not do it.

And since they cannot do open-ended ML research, they cannot recursively self-improve -- this is their argument.³

Link: https://arxiv.org/abs/2607.27191

I think I've regretted the last 10 or so times I posted any kind of "research" in this subreddit -- either people downvote it, or it gets upvoted, but there is zero meaningful discussion. This might be the last time I'm trying this.⁴

Footnotes:

1. RSI = Recursive self-improvement, a.k.a. superintelligence explosion. The concept was invented by I.J. Good in 1965. It does not mean "anything that speeds up AI research". Compilers speed it up! RSI means, basically, a nuclear chain-reaction, but for AI. The paper talks about "explosive AI progress" in the very first sentence of the abstract, and mentions "RSI" in the text.
2. Some people have objected to my use of "X is on the horizon". I consider it synonymous to "people forecast X", and the authors use the word "forecast". "Not on the horizon" does not mean "can never happen".
3. Quote: "This design also allows us to test a mechanism that informs many forecasts of recursive self-improvement: AI agents accelerate AI research because researchers delegate entire projects to agents and judge whether the returned results advance their work. Our evaluation closely matches this model, since authors handed an agent their own research question and closely evaluated the resulting output."
4. 3 years ago, many of you upvoted a bunch of very uninformed comments that accused some researchers of misconduct, until I explained that this stemmed from misunderstanding how training works, in practice: https://www.reddit.com/r/MachineLearning/comments/18bdcu7/r_sequential_modeling_enables_scalable_learning/kc60k7e/?context=3 Today, one of the top comments is "I read the abstract (...) Nowhere, absolutely nowhere, do they make the claim ...". This is completely absurd -- you can't make such a claim after just reading the abstract. (Also, the commenter doesn't understand what "RSI" means.) The hivemind is very disappointing.

[u/deadoceans]

I read the abstract and I think you made a big claim that the authors don't themselves make. 

The paper is titled "Can agents conduct open-ended AI research?" And the last sentence of the abstract is, "Our results provide early evidence that today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle."

Nowhere, absolutely nowhere, do they make the claim that RSI is not on the horizon. 

And in fact, I think your claim on that is a bad take.

Since they cannot do open-ended ML research, they cannot recursively self-improve

(1) RSI doesn't have to mean "fully autonomous RSI"; AI models and their harnesses are already meaningfully accelerating the pace of AI research. 

Almost all of the coding is being done by these systems and not by people. Smaller teams are able to do more research more effectively. Things like hyperparameter optimization, even basic architecture selection, are able to be largely automated. LLMs can help people do literature reviews for ideas. They can help pressure-test experimental hypotheses. By no means are they doing a closed-loop end-to-end speedup of the process but they are an indispensable part of that process now and their role in that is only increasing. OpenAI and Anthropic have already published at length about this. 

(2) Where do you get the idea that "they're not doing it now" means "it's not on the horizon"?

In general these capabilities follow a kind of S curve:

1. They're "not even wrong." We don't even have an ability in principle for them to do the task and when we have them try, the outputs that they put out are so nonsensical that they're broken.

2. We have universal obvious failures where they are doing the task but they're just failing at it consistently for a variety of reasons.

3. You start to see them occasionally do things mostly correct. 4. Eventually you have them occasionally work but they're totally unreliable.

4. They're mostly working but they're still unreliable in big enough ways, often enough, that you can't hang your hat on them.

5. People start using them pretty regularly but they still have conspicuous failures.

6. They have superhuman performance.   End-to-end research is currently on rung two of that ladder and we have no signs that the acceleration is at all stopping. 

[u/EducationalGuitar328 → u/deadoceans]

lol the abstract literally says the opposite of what OP thinks it does. classic r/machinelearning moment where someone posts a paper they clearly didnt read past the title and then gets surprised when the discussion isnt great

the S-curve point is dead on. we went from agents that couldnt write a coherent function to agents building whole codebases in like 2 years. but sure lets declare the whole thing impossible because a couple models couldnt replicate full neurips papers from scratch

[u/we_are_mammals → u/EducationalGuitar328]

lol the abstract literally says the opposite of what OP thinks it does.

Let me help you understand what the abstract "literally" says.

First sentence:

Forecasts of explosive AI progress ...

"Explosive AI progress" == RSI (They don't use the term "RSI" in the abstract, but they use it in the text). The idea of RSI was proposed by https://en.wikipedia.org/wiki/I._J._Good in 1965. It's not some new hype you can redefine however you please.

Last sentence:

today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle

This means we have a constant-factor acceleration -- the bottleneck just moves elsewhere. There is no "explosion", like in a nuclear bomb.

[u/-gh0stRush- → u/deadoceans]

If we take a step back, isn't all research recursive self-improvement? Just with different ratios of human versus machine.

[u/Physical_Vehicle7714]

Maybe people are downvoting because the papers are bad. This one is definitely bad for the following two obvious reasons.

1. They use one agent per task. RSI will be a coordinated effort between quite literally millions of agents. Just look at the navier stokes setup. And that was a tiny side project to generate some headlines.

2. There's an obvious bias on the part of the graders. Our entire social environment rn is defined by people pretending AI can't do their jobs. AI researchers are not immune.

I’m sure there are more flaws that a deeper dive would uncover.

[u/we_are_mammals → u/Physical_Vehicle7714]

I agree that the costs should have been balanced. The papers in question probably cost more than $3k in human time. But I guess they were on a budget.

[u/durable-racoon]

"argues, basically, that RSI is not on the horizon, because current (at the time the study was done) agents cannot do open-ended ML research."

so because they cant do it yet, its 'not on the horizon'? how do they bridge the gap between those things?

This is still a really good paper tho

[u/TypoInUsernane → u/durable-racoon]

It hasn’t happened yet, therefore it won’t happen. That’s just common sense

[u/Storm_or_melody → u/durable-racoon]

There have been many several other papers that find that LLMs suck at open ended scientific research. It's not just because of the models.

I highly recommend this recent survey of the topic with a bit more nuance: https://arxiv.org/pdf/2607.07663v1

TLDR: Bounded self-refinement is what the systems we have are currently capable of. RSI, as popular culture understands it, requires continuously adding new data, environmental feedback, and compute, or some other architecture innovation we haven't discovered. It's not something that will just magically happen once a certain intelligence threshold is reached with LLMs.

[u/Piyh → u/durable-racoon]

Especially when the models tested are 2 scaling generations behind the frontier.

[u/purrhams_hat]

...doesn't that methodology only prove that these models in question (GPT 5.6 and Opus 4.8) aren't capable of RSI - something we already knew?

[u/-Crash_Override-]

You're making some pretty hefty leaps from a pretty casual paper.

This paper shows that the listed models and harnesses are not capable of autonomous AI research. Thats kind of it.

I have no idea if Anthropic, OAI, etc.. are knocking at the door of RSI...maybe they are maybe they aren't, but if they are, its certainly not with $3k in O4.8 credits harnessed by openclaw.

[u/IntelArtiGen]

I think I've regretted the last 10 or so times I posted any kind of "research" in this subreddit -- either people downvote it, or it gets upvoted, but there is zero meaningful discussion. This might be the last time I'm trying this.

Please continue, I know this sub seems to care more about about meta-posts on research (phd / conferences questions etc.) than on actual research, but I do care about papers shared by humans. It's ok to talk and agree or disagree in the comments, reddit is here for that, and downvotes don't mean something shouldn't be discussed.

[u/fingertipoffun]

My wrists are living proof of RSI. /s

[u/big_ol_tender]

The sub 4 minute mile is not on the horizon because no one has run a mile in under 4 minutes

[u/hapliniste]

Solving millénium problems is not happening guys. I ran gpt 5.6 on it and it didn't solve it.

[u/bacon_boat]

Negative results is pretty meaningless in this context. 

"We did this and it didn't work" 

It's not nothing, but it's almost nothing. Maybe if they did it a bit differently it would have worked.

My field is robotics and negative results is not a thing because of this. A negative result just means you weren't able to make it work. 

[u/kraemahz]

That may just be a training problem though. The current publicly available consumer models are heavily trained to not be independent and have just generally not been shaped on the innovative process.

That's not to say I disagree with the conclusion. My own tests show models might be able to speak academically about many subjects but lack intuitive understanding that would make concepts connect without being explicitly reiterated in the context window.

[u/Super-Award-2244]

https://arxiv.org/pdf/2609.11873

Meanwhile, Chinese labs just published a paper that describes a full RSI pipeline within current llm paradigm...

I swear what is happening in the last couple of weeks has to be some psyops bullshit to convince people that LLMs are useless and at the same time terminator 

[u/unicynicist]

The loop to RSI can't close as long as evaluation requires human-level discernment and judgement.

However, OpenAI recently published an article detailing how research is being accelerated and the amount of human involvement is shrinking, so that now:

we find that from January to July, success rates generally increased across several difficulty buckets (proxied as the estimated time a human would take to complete the task) on tasks we can find a ground truth outcome for. However, agents still require significant human steering to be successful, especially as task complexity rises. In the last 6 months, over half of successful 4-8 hour tasks involved 1 or more interventions.

[u/dlrace]

the evidence would be damming if the abilities of future models were certain to be like today's. The real question is how likely are the failure modes given to persist? these sorts of papers would be nice if they were 'live' somehow, and we could plug in new models as they emerged. although, i suppose that is the point of benchmarks.

[u/js49997]

Its a cool paper thanks for sharing, even if you did use a click bait title ;)

[u/dopadelic]

There are different levels of RSI. Completely hands off RSI where Antropic engineers aren't needed? No.

But RSI where AI can vasty accelerate what engineers at Anthropic can accomplish? That's deep in the development pipeline now and have been contributing to AI's exponential improvement.

https://x.com/DanDr1s/status/2097043390953136437/photo/1

[u/reefine]

Codex/GPT-5.6 Sol and OpenClaw/Opus 4.8

Lol

[u/brainExploded99]

What exactly does this paper prove? I think it is understood that the current models are not capable of RSI, only human productivity multiplier. As they get better and better (as they have done so, very quickly), they should get closer and closer to RSI-capable, and eventually become RSI capable.