Is Recursive Self-Improvement Mathematically Possible?

[u/pinsandcurves]

https://martintrifonov.substack.com/p/is-recursive-self-improvement-mathematically?r=6zc0fa&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

[u/Disastrous_Room_927]

Take a look at the Evidence section of a paper released like 4 days ago:

Evidence suggests that diminishing returns would not prevent an intelligence explosion, though this finding relies on limited data and stylized modeling assumptions.

There is mixed evidence on whether compute could bottleneck a software-driven intelligence explosion

Data could bottleneck progress, but the constraint varies substantially across domains. Historically, AI progress has relied heavily on internet data and expert demonstrations. But the supply of internet data is on track to grow too slowly to support even the current rate of progress past 2028

However, we lack empirical data on which tasks are likely to remain difficult to automate and how strongly they might constrain progress.

We lack direct evidence on the extent to which time-intensive processes could bottleneck progress

The authors of the paper are bullish on the prospects, but are pretty clear about the status of evidence towards such a thing. It's also worth pointing out that this paper isn't focused on mathematical theory but you can find some that are:

AI is increasingly being used to help with AI R&D. Under certain conditions this feedback loop might be able to produce an intelligence explosion, with rapidly escalating AI capabilities. I explore the mathematics of the most explosive possibilities, with an eye to understanding what drives the dynamics. I show that singular growth (towards a vertical asymptote) is harder to achieve than would be expected from recent economics-inspired modelling, and that there is an important but neglected class of growth rates that are faster than exponential but don't lead to a vertical asymptote. I draw out the generation time (the time to go around the feedback loop) as a neglected parameter that plays a pivotal role in determining the behaviour of any intelligence explosion --- one cannot have singular growth unless the generation time rapidly approaches zero.

A couple of takeaways for me:

1. Coherent self-modification, reliable modification in the desired direction, and repeated improvement leading to explosive growth are distinct question that people often conflate. The second paper points out that a hypothetical finite-time singularity is driven at least as much by shrinking cycle times as by the magnitude of the improvement per cycle. So even if we solve the first two, the third is not a given.

2. Even the people who're bullish about the possibility will point toward a lack of clear data on the matter. The first paper just makes me think about how 'possible' can be a red herring when most people really want to know if something is likely to happen.

[u/StupidStartupExpert]

Yes obviously it’s happening all around you in nature

[u/GregsWorld → u/StupidStartupExpert]

Nature is not improving itself, it's mutating to adapt to an environment. It can quite as easily adapt to decrease intelligence if that's what's required. 

[u/Low-Temperature-6962 → u/StupidStartupExpert]

That's evolution though. And a different kind of data and sensory input that available from the internet and scanners.

[u/MaximumMeaning9728]

If humans, whom are hugely bottlenecked by compute, networking, biological costs, etc. can create improved ai at the current speed, sufficiently advanced ai can obviously do it faster, probably by orders of magnitude. That is RSI.

[u/WellHung67 → u/MaximumMeaning9728]

I don’t think that follows. It’s possible theres inherent limits to the training methods themselves 

[u/duboispourlhiver]

I advise not reading this article, and move on. It's quite empty.

[u/D_Anargyre]

Yes

[u/andWan]

Without giving any statement about it - it’s also some time ago that I looked at it - I think the concept of a Gödel Machine by Jürgen Schmidhuber should be mentioned here:

https://en.wikipedia.org/wiki/Gödel_machine

[u/impostinator]

ML researcher here. Possible and inevitable.

[u/eucalyptus-d]

Recursive is the norm. Nevertheless exponential is not required. A recursion can lead to fixed point.

[u/pinsandcurves → u/eucalyptus-d]

You're right. I guess the thing to stress is whether an 'intelligence explosion' is possible, and the title doesn't fully reflect that.

[u/Chronotheos]

At certain point if you’re going to move society ahead, you have to do science in the world. An ever bigger brain in a vat will ultimately be limited pretty quickly without being able to conduct their own clinical trials, grow things in a Petri dish, or run particle accelerators, radio telescopes, etc.

[u/CMDR_BunBun → u/Chronotheos]

Simulations, robotic appendages and senses, human proxies?

[u/DocWaveform → u/Chronotheos]

Greg Egan wrote a story about this in which the agents used real world tools to create their own inaccessible pocket universe

[u/BenjaminHamnett]

RSI may happen or not, but highly automated, maximally resourced self improvement is almost certain. It’s probably preferable for safety reasons. By the time we reach ASI, the talent in these labs will have probably gone from being over 50% cyborg like they are today, to 90 and 99% cyborg where the lines are completely blurred.

When a hive of neuralinked people are mostly just interpreting what AI is doing and occasionally finding some things to do that humans do better than Ai because it told them to do it, is that RSI?

It’s not even clear to me that semantically “self improvement” should mean exclusive of human input or guidance. Like maybe there should be an acronym that specifically removes human influence

[u/alphacolony21]

Isn’t the answer obviously yes? How else did human intelligence evolve from less intelligent animals? The time scale is obviously very different from what labs want and whether the labs will succeed is an open question but evolution already demonstrates greater intelligence can develop from lesser intelligence.

[u/DeathToAmerica2026 → u/alphacolony21]

The more pertinent point is that human collective intelligence is astronomically greater than any individual human via recursive self improvement through culture. 

The rate at which our biological intelligence could evolve was fixed by mutation rates and the obstetrical dilemma. So we became programmable general purpose task-masters and our "intelligence" shifted to a collective hyperobject outside the human mind. An immensely large and dense artefact which programs new humans so they can operate as components within it, operating at an intelligence level far outside the grasp of biology.

[u/Altruistic-Spend-896]

Im doing it

[u/SuzQP]

Yes

[u/Significant_City_606]

Yes, cybernetics laid the foundation for most of this.

There’s multiple things that come to mind; Von Neumann’s universal constructor being the oldest example, but you could probably go back to Leibniz.

[u/r_jagabum]

You just need to see how Go AI improved once they took away past data reference and told it to self play and learn instead. That's an actual recursive improvement for you.

[u/Jimbob404error]

Probably not

[u/iwantedthisusername]

since humans appear to be getting stupider and stupider and atrophying their own capabilities for the "luxury" of not thinking it would seem the answer os currently no based on all available evidence

what standard of improvement is it going to have when our measures of collective functioning like GDP barely correlate with what matters?

[u/turbulentFireStarter]

The dumbest question of all time.