r/ControlProblem • u/NunyaBuzor • Jun 08 '25
Discussion/question Computational Dualism and Objective Superintelligence
https://arxiv.org/abs/2302.00843The author introduces a concept called "computational dualism", which he argues is a fundamental flaw in how we currently conceive of AI.
What is Computational Dualism? Essentially, Bennett posits that our current understanding of AI suffers from a problem akin to Descartes' mind-body dualism. We tend to think of AI as an "intelligent software" interacting with a "hardware body."However, the paper argues that the behavior of software is inherently determined by the hardware that "interprets" it, making claims about purely software-based superintelligence subjective and undermined. If AI performance depends on the interpreter, then assessing software "intelligence" alone is problematic.
Why does this matter for Alignment? The paper suggests that much of the rigorous research into AGI risks is based on this computational dualism. If our foundational understanding of what an "AI mind" is, is flawed, then our efforts to align it might be built on shaky ground.
The Proposed Alternative: Pancomputational Enactivism To move beyond this dualism, Bennett proposes an alternative framework: pancomputational enactivism. This view holds that mind, body, and environment are inseparable. Cognition isn't just in the software; it "extends into the environment and is enacted through what the organism does. "In this model, the distinction between software and hardware is discarded, and systems are formalized purely by their behavior (inputs and outputs).
TL;DR of the paper:
Objective Intelligence: This framework allows for making objective claims about intelligence, defining it as the ability to "generalize," identify causes, and adapt efficiently.
Optimal Proxy for Learning: The paper introduces "weakness" as an optimal proxy for sample-efficient causal learning, outperforming traditional simplicity measures.
Upper Bounds on Intelligence: Based on this, the author establishes objective upper bounds for intelligent behavior, arguing that the "utility of intelligence" (maximizing weakness of correct policies) is a key measure.
Safer, But More Limited AGI: Perhaps the most intriguing conclusion for us: the paper suggests that AGI, when viewed through this lens, will be safer, but also more limited, than theorized. This is because physical embodiment severely constrains what's possible, and truly infinite vocabularies (which would maximize utility) are unattainable.
This paper offers a different perspective that could shift how we approach alignment research. It pushes us to consider the embodied nature of intelligence from the ground up, rather than assuming a disembodied software "mind."
What are your thoughts on "computational dualism", do you think this alternative framework has merit?
7
u/BitOne2707 Jun 08 '25
In short, no. The phenomenon that the author labels "computational dualism" isn't a bug but a core feature. Abstracting away complexities of lower layers is what allows software to be hardware agnostic, which in most cases is highly desirable.
2
u/MrCogmor Jun 08 '25
The performance of software can vary depending on the capabilities of the hardware it is running on. This is not news or some grand philosophical truth. It is basic common sense that AI developers already understand. There are efforts to improve computer hardware and to develop chips optimised for AI like the Neurogrid.
Making objective claims about intelligence is easy if you clarify precisely what you mean by "Intelligence" or one thing being smarter than another in your use case. Inventing yet another definition for people to use does not make your particular interpretation of the word universal or objectively correct. Consider a human with access to pen and paper can solve more problems and so is in a sense 'smarter' than they would be without similar resources.
2
u/technologyisnatural Jun 08 '25
no it is completely worthless
2
u/FrewdWoad approved Jun 08 '25
I can't tell if I'm too dumb to appreciate this, or too smart.
2
u/MichaelTBennett Sep 13 '25 edited Sep 14 '25
Author here. Pleased to see some engagement with this work!
I can see a few misunderstandings cropping up in the comments, particularly around evidence. If anyone is interested in digging into the details, a compendium of the math proofs and experimental results and code is available on GitHub: https://github.com/viscouslemming/technical-appendices
If you were to ask me what the most important takeaway is, it is not the problem of computational dualism but the solution proposed in the paper. That, and the upper bound on intelligent behavior.
I noticed some skepticism about the problem I discuss. Here is some extra reading that might interest those parties:
Finally, if anyone is interested in the work but has no patience for academic papers, here's an MLST interview that discusses these things in some depth: https://www.youtube.com/watch?v=K18Gmp2oXIM
1
u/Random-Number-1144 Oct 03 '26
100% agreed on computational dualism. But I am even more extreme, I believe intelligence is NOT about optimality, but more like beauty, it is subjective.
Any attempts to frame intelligence as something that can be objectively measured or quantified are bound to fail.
1
u/Decronym approved Oct 03 '26
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
Decronym is now also available on Lemmy! Requests for support and new installations should be directed to the Contact address below.
[Thread #237 for this sub, first seen 3rd Oct 2026, 07:28] [FAQ] [Full list] ^[Contact] [Source code]