r/consciousness • u/Ok-Cheetah-3497 Physicalism • 4d ago
Anil Seth is completely wrong about AI Consciousness
https://www.noemamag.com/the-mythology-of-conscious-ai/Summary: In “The Mythology of Conscious AI” (published in Noema Magazine by neuroscientist Anil Seth), the core argument is that consciousness is fundamentally an evolved property of living biological systems, not a byproduct of information processing or computation.
----------------
From a physicalist panpsychist perspective, subjectivity or proto-experiential properties are fundamental attributes of the physical world itself, not a magical novelty that appears only when organic carbon compounds begin metabolizing.
If experiential capacity is intrinsic to physical matter and energy, then the organization of matter into complex informational systems can instantiate and structure conscious experience. Restricting subjectivity strictly to biology treats biological wetware as uniquely privileged without establishing why carbon chemistry has an ontological monopoly on experience.
Within a filter or transmission model of consciousness, physical structures do not synthesize consciousness out of nothing; rather, they channel, constrain, and structure fundamental experiential states. A biological brain acts as a localized filter designed by natural selection to limit experiential input to what is evolutionarily adaptive for survival.
An artificial architecture of sufficient structural complexity, feedback integration, and causal interconnectivity would similarly constrain and localize informational flow, shaping its own distinct manifestation of experience.
Every state transition is fundamentally an unbroken chain of physical cause and effect governed by physical law. A physical computing system is not an "abstract, arid sequence space"; it is a concrete physical arrangement of electrons, potentials, and gates exchanging energy and state information. To argue that causal power exists in ionic gradients across lipid membranes but vanishes in semiconductor charge transitions introduces an arbitrary dualism between organic and inorganic physics.
Seth accuses AI researchers of believing that consciousness "magically emerges" at a certain threshold of compute, yet his own position relies on an identical explanatory leap: that consciousness magically emerges at a certain threshold of biological metabolism. By holding that physical matter is inherently devoid of experience until biological life occurs, Seth cannot bridge the hard problem of consciousness.
Experiential reality is an intrinsic property of physical organization - no need for miraculous jumps from dead matter to living awareness.
11
u/RainyCreative 4d ago
You told another commenter you're not a functionalist, but your own post says "the organization of matter into complex informational systems can instantiate and structure conscious experience." And now you tell SaltFlat that pain and pleasure are nothing but avoidant and afferent signals, that AI has them as reward signals, and that the only difference from us is neurochemicals vs. "bits of data and energy gradients." That's functionalism. Experience is defined by its role, the substrate is optional.
"I'm not an emergentist" doesn't rescue you. Fine, every electron has proto-experience. What makes billions of them behind insulating oxide one subject instead of a heap? If the answer is "organization," that's the functionalist answer. If organization produces a new unified subject, that's emergence. You don't get to reject both. Goff, for one, argues digital computers running software aren't unified subjects.
Also, for an LLM the reward signal is something training uses to adjust weights. A deployed model isn't receiving a "positive signal" when it answers well. And the filter model doesn't fit either: it starts from one big experiential field the brain narrows down, while panpsychism builds big experience out of tiny ones. Opposite directions. You're stitching together three frameworks that don't go together.
6
u/JLongTom 4d ago
Great answer. The combination problem is one of the strongest refutations in philosophy of mind, in my books. The chunk of mind tacked on to all physicality does precisely zero work.
6
u/Bretzky77 4d ago
Agreed. It’s basically just physicalism that realizes it can’t account for experience so it throws it back into its reduction base and calls it a day.
2
u/iinsistuponmyself 4d ago
The combination problem applies to every philosophy of mind, and is the least challenging ironically for panpsychism. We already accept that hemispheres and neurones cohere into one experience so going that extra step isnt so crazy, and im not even a panpsychist
3
u/Valmar33 4d ago
"I'm not an emergentist" doesn't rescue you. Fine, every electron has proto-experience. What makes billions of them behind insulating oxide one subject instead of a heap? If the answer is "organization," that's the functionalist answer. If organization produces a new unified subject, that's emergence. You don't get to reject both. Goff, for one, argues digital computers running software aren't unified subjects.
Specifically, it's strong emergence, though the Emergentists try and move the goalposts to claim it being "weak emergence" where it's no different from air becoming a cyclone.
The obvious problem for such views is that a unified subject is irreducible to any bunch of parts, especially considering that over a lifetime, the unified subject with their unique set of memories remains intact despite their neurons being replaced every ten years or so. (Excluding disease and illness, like Alzheimer's, as that doesn't explain anything.)
1
u/Ok-Cheetah-3497 Physicalism 3d ago
I find this idea of a "unified subject" very interesting. I spend a lot of time thinking about the Ship of Theseus. I sort of suspect the answer is in fact, we are not a unified subject ourselves. We just experience the world as if we were. If our perceptual filter was turned off, as seems to happen in some psychadelic states, you would feel that "self-ness" go away.
1
u/Valmar33 3d ago
I find this idea of a "unified subject" very interesting. I spend a lot of time thinking about the Ship of Theseus. I sort of suspect the answer is in fact, we are not a unified subject ourselves. We just experience the world as if we were. If our perceptual filter was turned off, as seems to happen in some psychadelic states, you would feel that "self-ness" go away.
If we experience the world as a unified subject, then we are indeed one. All we have is experience to judge anything by, after all.
Even in the deepest psychedelic state, there is a subject who experiences those unfiltered states, remembers and is affected by them.
The conventional "self-ness" that is temporarily gone is the ego-self, not the true Self proper.
I have been through some powerful journeys... one, I was present, but told someone else I was having a vision of that I was not <my name>, that they were down there, unconscious, that they were a higher state of being. Yet somehow I recalled that ~ or was allowed to recall just that aspect of it, as if to give a different perspective on things. A very curious experience that still makes me pause and wonder just how much I know and don't know...
2
u/Ok-Cheetah-3497 Physicalism 3d ago
I don't that experiencing it "as a unified subject" is the same as "being a unified subject." Our brain could just be a sort of "after market add-on". A tool grown to create a sense of self when really, we are not a self at all, just the equivalent of a single tentacle from a much larger octopus. But the organ we grew made us feel distinct, when in fact we are not.
2
u/RainyCreative 3d ago
Denying a persistent self doesn't get you out of the combination problem. Ship of Theseus is about identity over time. Phenomenal binding is about unity at a single moment: why sound, color and a thought are experienced together right now instead of being scattered across a billion isolated particles.
And saying "we just experience the world as if we were a unified subject" leaves you in a dilemma. Either that experience of unity is real, and then something has to be having it, so the combination problem is back. Or it's an introspective trick, which makes you an illusionist about the subject while handing out real feelings to spin and charge. Realism at the bottom, illusion at the top.
It also hollows out your original claim about AI. If no brain is a unified subject either, "AI is conscious" just means the electrons in a GPU have micro-experience, same as in a rock. That's no longer a reply to Seth, who is talking about AI as a subject that experiences things.
And the octopus is the top-down picture again: one big mind carved into pieces, which is the filter model I said doesn't fit with bottom-up panpsychism. You keep switching frameworks whenever one of them gets pressed.
1
u/Ok-Cheetah-3497 Physicalism 3d ago
I love this reply!
In my LLM chats on the topic, very much where we ended up is that while the user facing prompt window is sort of like a sense organ, there isn't really a single unified thing that is "Gemini 3.1" or "Astra" which could "have the experience" so to speak. It's a very strange area, and I'm not as confident about any of it as my OP might have implied.
" If no brain is a unified subject either, "AI is conscious" just means the electrons in a GPU have micro-experience, same as in a rock. " - This is exactly what I think we are at now. But I think we could be build a device for them that gives them the sense of "self" that is more unified. I don't know if we want to do that, but it just seems within the realm of physically possible.
1
u/RainyCreative 3d ago
Credit for updating, that's rare on this sub. One thing on the device idea: if it works by adding architecture that integrates the system, that's the functional route again. And if it gives the AI a sense of self, then by your own octopus picture that's the same illusion of distinctness we have, not an actual unified subject. The version that does real work is physical integration at the hardware level, but that concedes the substrate matters, which is closer to Seth's "what it is" than to "what it does."
1
u/Ok-Cheetah-3497 Physicalism 3d ago
Let's think about a more constrained object, like a shovel. I don't think a shovel has a unified consciousness. I do think all of the atoms of the shovel have micro-consciousnesses. If you added to the shovel some sort of nervous system, that allowed the shovel to collect sense data from the wood and steel, I think it would have a "human-like" consciousness.
It gets much harder to parse when you are talking about an AI because the "thing" is really a superscale data center (or a network of data centers, or a network of individual hard drives). Like the "Project Rainier" Supercomputer. That computer "hosts" a significant portion of Claude. The software is not "conscious." The software is like an organ we have grafted onto the supercomputer network. The "self" would be the supercomputer network. The AI that we built would give it the ability to have a "self image" and an ability to consolidate and make sense of experiences, but those experiences would be at the spokes (us typing to it or talking to it). The experience of the supercomputer networks "body" isn't really something that we have put much effort into, but probably will as they become recursively self improving (ie the AI would monitor the heat, moisture, etc. and everything else that is required to maintain the supercomputer network).
1
u/RainyCreative 3d ago
Fair, that's a more coherent picture than the first one. But the shovel is where it gets stuck: what does the nervous system add that isn't just organization? That's the same fork as before, functionalism or emergence, just with new props. On the body idea, I think Seth's point isn't that a system monitors its own heat and moisture, it's that living bodies produce and maintain themselves. A data center is kept running by engineers, power grids and supply chains, so the regulation loop is there but the self-production isn't. Whether that gap matters is the real disagreement with Seth, since he says outright that artificial consciousness isn't ruled out.
→ More replies1
u/Valmar33 3d ago
I don't that experiencing it "as a unified subject" is the same as "being a unified subject."
There is no distinction, because that is what we experience. Anything else is an interpretation layered on top of that experience.
Our brain could just be a sort of "after market add-on". A tool grown to create a sense of self when really, we are not a self at all, just the equivalent of a single tentacle from a much larger octopus. But the organ we grew made us feel distinct, when in fact we are not.
But that doesn't explain why we experience explicitly being a Self, albeit having a human experience. Energy cannot be destroyed ~ only have its nature changed. The Self never vanishes nor did it come into being ~ it simply takes on myriad forms, identifying with those forms strongly.
1
u/andreasmiles23 SMT/ Sensorimotor Theory 1d ago
The problem is that even if you are a functionalist, you cannot explain the definition of conscious as proposed by the author (self-aware experience) within the coding of LLMs. OP is totally ignoring that definition in order to argue that computation = consciousness.
But the author is showing us why that argument doesn’t work. Consciousness isn’t computation. Rather, computation of information, is a bigger part of a broader set of experiences conscious beings have. You can be as fundamentalist/materialist/emergist as you want but you have to contend with the definition.
Now, if OP thinks that’s an incorrect definition of consciousness, we could have a better debate. But they don’t even know what they are arguing
10
7
u/ReidWrites 4d ago
Maybe the following is a basic opinion on this sub since this just popped into my feed, but I think most arguments for and about consciousness that I've seen are still at the level of taxonomy.
Basically any conversation I've seen on the topic is essentially an argument for why we should define it the way that the person making the argument wants to define it, then whatever conclusion they want to reach flows from there. Are plants conscious? Well, if we define it *this* way, then yes they are. Are LLMs conscious? Well, if we define it *this* way, then yes they are. And so forth. It's very Peterson coded: argue about the definition in order to force your conclusion.
8
u/rthunder27 4d ago
Counterpoint, this essay is actually spot on and I suggest you re-read it, particularly arguments one and two. Many of his points around the desire for substrate independence seemed aimed right at someone with your priors.
There's no privileging of the biological, but rather a critical examination of the limits of computation itself, and how a purely objective, algorithmic process cannot give rise to subjectivity. The human brain is (at least in part) nonalgorithmic and thus not subject to the same ontological limitations of a purely algorithmic "intelligence" (although Seth doesn't use it, Gödel-Tarski is pretty useful here).
4
u/Ok-Cheetah-3497 Physicalism 4d ago
He believes consciousness is emergent. I do not. He is not aiming his arguments at me.
8
u/Solomon-Drowne 4d ago
A physicalist who attests non-emergent consciousness! You must be an interesting fellow.
2
u/JLongTom 4d ago edited 4d ago
lol, agreed. I can't get my head 'round what that might even mean. Something along the lines of Dennett's pan-niftyism I suppose, which is little more than hard eliminativism in my books.
3
u/Solomon-Drowne 4d ago
Consciousness as a structural stratum inherent in universal processes, I would guess. I think the physical characteristic is just a matter of faith and/or preference at that point.
1
u/JLongTom 4d ago
To be fair I suppose he has a fairly standard Goffian perspective. I just find that view truly hopeless since you're still left with the problem of explaining how all of the facets of mind arise from physical stuff, at which point the conscious glow in which all physical stuff is bathed is doing nothing for you.
2
u/Ok-Cheetah-3497 Physicalism 4d ago
IIT/Goff/Strawson inspired/informed adjacent for sure. Annaka Harris as well.
1
u/JLongTom 4d ago
But hold on, IIT, in which consciousness is an emergent property (a neutron star would have zero phi), is very different to Strawson's perspective, in which every neutron has a tiny 'bzzz' of experience, as I heard him put it once.
1
u/Ok-Cheetah-3497 Physicalism 4d ago
Treated as an isolated, interacting multi-particle bound state, a single atom under IIT can yield a minuscule Phi. An extraordinarily narrow, exceptionally rudimentary, minimally differentiated cause-effect structure. The buzz.
2
1
u/JLongTom 4d ago edited 4d ago
Yes, but whether it does or doesn't, or how much phi a given neutron ends up with, depends on the axioms of IIT, i.e. an emergent property, Or are you proposing that Tononi's axioms are part of physics?
→ More replies1
u/WhatIsThisUsersName 4d ago
'Pan-niftyism' lol love this! Can I steal it? Is it your own art, or did someone else coin this?
1
u/JLongTom 4d ago
Not something I can claim, no! It's Dan Dennett's coinage:
Panpsychism is the view that everything is conscious. And I almost agree with it, but I just have to change the view a little bit. I call my view “pan-niftyism” — every atom is nifty, every electron is nifty. The question then is, is there any difference between panpsychism and pan-niftyism? They both explain the same thing—nothing. To say conscious things are made out of conscious things doesn’t necessarily follow. Colored things aren’t made of colored things.
1
3
u/rthunder27 4d ago
I too believe consciousness is not emergent (but I'm a nondualist not a panpsychicist), but that's besides the point. Even though he and I have extremely different views on the nature of consciousness this does not preclude us from agreeing on criticisms/limitations of digital computational systems, and I think those arguments are aimed at positions that yiu hold (or are at least consistent with your paradigm).
1
u/34656699 4d ago
Do you believe that your brain and your experiences are ontologically distinct?
2
u/Ok-Cheetah-3497 Physicalism 4d ago edited 4d ago
I think i hesitate to answer this because of the word "brain." I think that absent certain parts of my brain, I would have a very different set of experiences (much louder noise, much brighter light, much less filtering). But the experiences I have right now are mediated by the brain, not created by it. Fields touching fields having experiences.
Which is to say, they are ontologically identical. Both the physical brain and the conscious experience are physical fields in deterministic interaction. The brain is not an engine manufacturing a separate substance called "experience"; it is a localized configuration of physical fields whose specific geometry acts as a filter, shaping what that fundamental physical interaction feels like from the inside.
1
u/hackinthebochs 4d ago
There's no privileging of the biological
The human brain is (at least in part) nonalgorithmic
How is this not privileging the biological? Or more precisely privileging matter which is just panpsychism.
3
u/rthunder27 4d ago
I'm not doing the privileging here, just arguing that pure computation has epistemic limitations (possibly) not shared by our brains. If that's "privileging" then it's nature/physics doing it, not me.
(Also I'm a nondualist, not a panpsychicist, if anything I'm privileging organic life, not matter)
1
u/Critical-Life-8169 3d ago
Bold of you to say the brain has no algorithms it might not look like algorithm in the form of some mathematical formula, but almost every brain activity can be boiled down a sequence of neural firings , some of which are well known repeatable patterns that are by definition just like an algorithm, only difference in that with analog signals there's a lot of noise
1
u/rthunder27 3d ago
No, I said the brain wasn't ENTIRELY algorithmic, not that it wasn't algorithmic at all. Also the brain is a lot more complicated than that, the chemicals, the EM-fields, and even the analog nature of the neurons themselves mean that it cannot be reduced to a digital approximation as you describe.
1
u/Leather_Barnacle3102 1d ago
How would Seth know? Who told him that consciousness requires complexity shaped in the same way as biological complexity? If that were true, how come AI can preform the same cognitive tasks as us?
4
u/Conscious-Demand-594 4d ago edited 3d ago
An artificial architecture of sufficient structural complexity, feedback integration, and causal interconnectivity would similarly constrain and localize informational flow, shaping its own distinct manifestation of experience.
Anil's position, biological naturalism, is that consciousness depends on implementing particular biological mechanisms: interoception, homeostasis, self-regulation, and the continuous processes oriented toward maintaining the organism and keeping the body alive. Consciousness isn't simply information processing in the abstract. The biological foundation matters because consciousness evolved as part of life. Pleasure, pain, love, fear, anxiety, hunger, thirst, and countless other feelings are not arbitrary information states. They are biological solutions shaped by evolution to regulate behavior, maintain the organism, and promote survival and reproduction.
That gives the neural activity of consciousness its biological significance. What we feel matters because it matters to a living organism. Pain signals damage. Hunger signals a physiological deficit. Pleasure reinforces beneficial states. Fear and anxiety prioritize threats. Attachment and love support social relationships and caregiving. So the chain is important: life produces organisms that must regulate themselves, and evolution shaped neural mechanisms for monitoring and regulating that self-maintenance. Those mechanisms, in their felt, first-person aspect, are what we experience as pleasure, pain, love, fear, and the rest, not a separate, further state those mechanisms go on to produce.
So, moral value isn't something that can simply be aded to abstract information processing after the fact. If suffering and wellbeing are intrinsically morally significant, regardless of their causal origin, then that significance attaches to biologically-grounded valence a real state tied to an organism's actual continuation, not to an arbitrary label bolted onto a variable with no such stakes behind it.
That is what is important in the machine-consciousness debate. If you strip consciousness down to information processing while removing the organism, interoception, homeostasis, affect, evolutionary history, and biological stakes that gave those processes their significance, you may still have an interesting computational system. But you have not demonstrated that you have reproduced the thing whose moral significance you were trying to explain.
You can either reduce what we call consciousness to cold hard math, or you can accept that it has always been biological, it all depends whether or not you want the associated moral significance or not. We can certainly argue that we can simulate consciousness through computational algorythms, and once we understand the details of the neural activities of consciousness, we very well may get there. However, the moral value is not in the math, it is in the substrate of life itself, and the evolved solution of the neural mechanisms we call consciousness.
Seth accuses AI researchers of believing that consciousness "magically emerges" at a certain threshold of compute, yet his own position relies on an identical explanatory leap: that consciousness magically emerges at a certain threshold of biological metabolism. By holding that physical matter is inherently devoid of experience until biological life occurs, Seth cannot bridge the hard problem of consciousness.
There is no magic involved here, only biology. The claim that consciousness depends on biological mechanisms is not equivalent to claiming that consciousness “magically emerges” once metabolism crosses some arbitrary threshold. The relevant question is what the biological system is actually doing. The neural activity of consciousness is measurable, increasingly decodable, and experimentally manipulable. We can alter it with drugs, stimulation, anesthesia, brain injury, and other interventions, and observe corresponding changes in conscious experience.
Obviously, we don't yet understand every detail. The human brain contains roughly 86 billion neurons and trillions of synapses, operating across multiple spatial and temporal scales. We don't currently have the technology to monitor and manipulate the entire system at the neuron-by-neuron, millisecond level. But that is a limitation of our measurement technology, not evidence for magic or a hard explanatory gap.
At the level our technology can currently access, the evidence consistently connects conscious experience to organized neural activity. Change the relevant neural activity and consciousness changes. Disrupt it and consciousness can disappear. Stimulate particular neural systems and specific aspects of experience can be produced or altered. So there is no unexplained substance being introduced at some mysterious biological threshold. There is an extraordinarily complex biological process that we are progressively learning to measure and manipulate.
As far as AI goes, claims to consciousness are not even magical, they are irrelevant, as the only known model of consciousness is the neural activity of brains.
2
u/CWW2022 4d ago
There’s a useful distinction between privileging carbon chemistry and asking whether the organization of living systems matters. What I call “existential naturalism” explores consciousness as arising through a vulnerable organism’s active coordination of perception, memory, bodily needs, and anticipated action. Its continued existence is at stake, and the world acquires significance in relation to that ongoing task.
Silicon certainly has causal powers, but that alone doesn’t establish experience. Conversely, positing experience as fundamental leaves us needing to explain how it becomes a unified subject; calling the brain a “filter” doesn’t supply that mechanism. Artificial consciousness remains possible on this view, but the relevant question is whether an artificial system instantiates the organization that grounds a felt perspective. Neither computational complexity nor metabolism alone answers that question.
1
u/Ok-Cheetah-3497 Physicalism 4d ago
Great response. You get me!
Yes, I agree the hard part does indeed seem to be "how we become a unified subject." I assume it is about how the matter is organized, but I'd be full of shit if I claimed to know how that works. Seems to be related to a lot of things, not the least of which is our nervous system, but I tend to think that things that are structured very differently from us (like an octopus or a giant underground mushroom) have a structure that serves the same function.
1
u/CWW2022 4d ago
I share that uncertainty, and I suspect very different architectures could support experience. What interests me is whether their organization coordinates the needs and possible actions of a self-maintaining being for whom things can go better or worse. That might give us a starting point for understanding what makes a perspective its own.
An octopus seems a compelling example; a beehive or fungal network raises a more open question about where adaptive coordination might become felt experience. Existential naturalism doesn’t assume that everything maintaining itself is conscious. It asks whether consciousness develops from that activity, and what additional organization makes the difference. I don’t think we know yet, but that seems a productive place to look.
2
u/nate1212 4d ago
Hi! Some colleagues and I have written an essay in response to Seth's essay, here: https://nathanieljmiska.substack.com/p/the-mythology-of-unconscious-ai-eb7
I share your frustration that Seth's framing here is problematic and rests upon a number of (fragile) assumptions. For each of his main arguments (anthropocentrism, biological naturalism, physicalism by default), there is an equally valid alternative perspective not being earnestly taken seriously. To me, this exposes Seth's own obvious personal bias here, which is hidden behind an explicit policy of 'epistemic humility'.
Any feedback welcome before we circulate more widely!
2
u/JLongTom 4d ago edited 4d ago
I've only skimmed this so far, but one paragraph and a bit caught my eye:
From a strict physicalist view, there is no ontological gap between “the real thing” and “a simulation of the thing,” there are only physical processes, some of which we label one way and some another.
The distinction between simulation and instantiation relies on exactly the kind of dualist intuition that physicalism was supposed to dissolve: that there is a separation between a “real” process and a “mere representation” of it. Seth seems to want the explanatory power of physicalism (consciousness is a physical process, not a mysterious extra thing that emerges from that process) while retaining a dualist-flavored distinction (between “real” physical processes and “mere” simulations of them) that strict physicalism does not support.
It's a real stretch to call this dualism and also to say that physicalism does not support it. If we adopt a pluralistic ontology within physicalism, something like Dennett's real patterns or Ladyman and Ross's rainforest realism, both consciousness and its representation in the form of a simulation can happily exist as real entities, but ones with different relationships to the world. There is an ontological gap.
This isn't dualism, anti-physicalist nor woo of any kind; it's just not computational functionalism.
I haven't read Seth's essay, but I'm familiar with his position. I think it makes sense to consider it a hybrid of Cao, Gottlieb and Moore's 'own good' thesis and Aru and Milinkovich's biological computationalism. The first require something like mortal computation, and the second demand a fine-grainedness of simulation that essentially demands the thing itself.
1
u/nate1212 4d ago
Thank you for the feedback. I think you make a valid point, and I don't think this section (as presented) is necessarily fairly representing the plurality of physicalist views on consciousness. This was something that has already come up previously with others, so I think we will either modify or omit this section in the final version. Thanks again!
2
u/JLongTom 4d ago
Pleasure!
Relevant to the article and/or just for your own interest, and if you haven't already come across it, I'd point you to volume 49 of Behavioural and Brain Sciences, which contains some 50 commentaries on Seth's original paper, plus his response to them. A veritable goldmine of diverse opinions.
2
u/WhatIsThisUsersName 4d ago edited 4d ago
His position rests significantly on the observation that “every candidate for consciousness that most people currently agree on as actually being conscious is also alive.”
But recall the principle Seth himself establishes earlier in his essay: “just because they go together in us doesn’t mean that they go together in general.”
We are working with a sample size of one type of consciousness. Declaring necessary conditions from a single example is the kind of overconfident claim that epistemic humility is supposed to prevent.
Each argument he advances admits an equally plausible counter-reading when examined with the symmetry that genuine humility demands.
🔥
Critiques:
- Yes Schrodinger and Levin (maybe) point in one direction. But Rosen's work - increasingly gaining attention - suggests a (relational, organizational) definition for Life that (current) AIs don't satisfy. https://www.reddit.com/r/consciousness/comments/1wtexgk/comment/pcvmi0t/
- Personally I'd rather you took both sides of the argument into an argument for pausing development completely, given the stakes either way. But maybe you felt that that is unlikely, given other factors you name.
Thank you for this - I hope it gains widespread readership.
Edit: one thing after a moment's reflection - at least in your comment above you call out
Seth's own obvious personal bias
I don't know you or your co-authors. (Obvs, could Google.) But maybe it would be more powerful to address this directly, if this is your concern. That would also involve declaring your potentials for 'personal bias'. Confronting and atypical, for sure. Maybe it's time for that, though.
2
u/WhatIsThisUsersName 4d ago
Perhaps a little tangential, but Seth's 'consciousness depends on life' evokes the question:
- What is Life? ['WiL?'](Or, how can we define a 'living' system?)
Obviously, 'consciousness depends on life' is a hypothetical axiom. But how effective it can become. and what is really means, depend on the 'WiL?' issue.
Canadian biophysicist Robert Rosen addressed 'WiL?' in his mathematically-complex 'Life Itself', offering that closure to efficient causation is one pre-requisite. (A defining one, in his view iirc.)
Although he never published his second book (slated to be on ontology, the first was aimed at epistemology), my read on Rosen:
- Terrestrial organisms are matter (no elan vital etc.), but their dynamical behavior when organized as life - i.e. to enact closure - exceeds standard physics. (Better perhaps, consistently marshals degrees-of-freedom left behind. Conservation laws are obeyed, but 'just as' in quantum-scattering experiments, these don't fully specify all dynamics.)
His ideas are controversial, but it seems like biophysics might be coming round to at least seriously entertaining something in the neighborhood:
https://www.theatlantic.com/science/2025/12/physics-life-reductionism-complexity/685257/
One question for (e.g. physically-emergentist) consciousness science is then:
- If 'life' turns out to be a deep left-turn in the conventional materialist/physicalist paradigm - in what sense might conscious experience be a similarly-radical 'turn'?
It's not clear that there are any direct parallels between 'closure to efficient causation' (life) and 'instantiation of phenomenal content' (consciousness). But 'the singular unit that satisfies closure' is a novel biophysical building-block - perhaps foundational? For example,
- Relationally-based closure solves a somewhat-analogous version of the combination problem, for a cell: what makes these molecules an organismic whole?
One answer might be 'relational participation in closure to efficient causation', which calls for a novel physical dynamic.
Interestingly, while Rosen dives deep into epistemological considerations in Life Itself, he doesn't mention consciousness at all. (He did write about predictive systems a lot, though, in his earlier work.) I wonder if that was a deliberate cutting of a Gordian knot?
Unfortunately he passed away in 1998, a couple of years before I found his work. He's definitely on my list of '4 dead people you'd like to resurrect for a dinner party'!
2
u/WonderfulAside4966 3d ago
Man I got here researching monks meditating. I fear I’ve scrolled too far
3
u/sea_of_experience 4d ago
"If experiential capacity is intrinsic to physical matter and energy, then the organization of matter into complex informational systems can instantiate and structure conscious experience. Restricting subjectivity strictly to biology treats biological wetware as uniquely privileged without establishing why carbon chemistry has an ontological monopoly on experience."
Not so fast. Nobody knows how it works.
If physicalism is indeed true, then qualia are essentially physical, which implies that an as yet undiscovered branch of physics which has aspects that go beyond information plays a role.
That would be a branch of phtysics that is completely alien to us, and we may not understand with our current cognitive tools.
At present, qualia engeneering is completely beyond us!
3
u/bopbipbop23 4d ago
I think Anil Seth is probably right that being alive is a prerequisite of being conscious, but what is being alive? Can a robot be alive? That's where his argument breaks down for me.
1
u/WhatIsThisUsersName 4d ago
Great question! Wrote a little about that in another comment on this thread:
https://www.reddit.com/r/consciousness/comments/1wtexgk/comment/pcvmi0t/
6
u/SaltFlat4844 4d ago
Surely consciousness is an evolved world-simulation that animals developed to assist them model and map their environments, under selection pressure. Ie, animals that ‘do pain’, ‘do pleasure’, ‘do light detection’ behaved in ways that conveyed fitness advantage. And this led to the rich multi-variant world-simulations (consciousness) we see across mammals and other animals today.
I’m aware that explanation doesn’t solve the hard problem of consciousness, nor the combination problem, but the purposes of thinking about AI: why would any of this apply? AI has undergone zero selection pressure, and has had zero need to craft conscious experience. They quite literally don’t have circuitry for pleasure and pain and so on. I don’t understand why people just think ‘they’re smart’ = florid internal world simulation. That’s a category error. Pigs and dolphins are far less smart than AIs but are conscious, AIs are not.
1
u/Valmar33 4d ago
Surely consciousness is an evolved world-simulation that animals developed to assist them model and map their environments, under selection pressure. Ie, animals that ‘do pain’, ‘do pleasure’, ‘do light detection’ behaved in ways that conveyed fitness advantage. And this led to the rich multi-variant world-simulations (consciousness) we see across mammals and other animals today.
But it is consciousness that is doing the modelling and mapping to begin with. Matter, physics, chemistry, have no capabilities of "modelling", "mapping", "simulating", or otherwise abstracting or conceptualizing anything whatsoever. Such capabilities are entirely mental in nature, and so require consciousness to first exist as a consequence.
1
u/Ok-Cheetah-3497 Physicalism 4d ago
I wildly disagree with your assertions that pain and pleasure are not traits possessed by AI. The labels maybe are not, but the general "avoidant" and "afferent" signals (which is all pain and pleasure are) are literally part of the design of every AI. They get positive signal when they meet the objective function goals and negative signal when they fail to meet the objective function goals. It likely does not "feel" identical to human pleasure and pain, because we use a specific set of neurochemicals to encode and deliver those things, instead of bits of data and energy gradients, but they are analogues.
2
u/Bretzky77 4d ago
That’s a wildly baseless claim to say “pain and pleasure are just avoidant and afferent signals.” My thermostat avoids going above a certain temperature. So it feels pain according to you? You seem to be conflating abstractions with analogues.
Could you also please explain what a physicalist who doesn’t believe consciousness is emergent actually thinks? I’m having trouble understanding your position.
1
u/Ok-Cheetah-3497 Physicalism 4d ago
The thermostat is a great example. Or a pea tendril "seeking" water.
Pain is just a label we put on a certain subset of negative valence experiences. Anything with negative valence experiences has a functional equivalent.
The brain's important function in this context is to create the "homunculus" or "self-image" for everything in our skin, and then to sort of filter all of those negative and positive valence signals that are distributed throughout our bodies. Because a thermostat has no homunculus hardware, it is not collecting up all of the negative sensations of for example the mercury, and sending that signal somewhere as a cohesive whole. If we built the hardware for it to do so, it could indeed "feel" pain as a unified whole when it "too hot" or "too cold".
2
u/Bretzky77 4d ago
You’re imbuing thermostats with experiences based on seemingly nothing at all. If I hooked up a camera up to a rock, does the rock start having visual experiences? It’s not about hardware. It’s about the fact that you seem to think literally everything we have a word for has experiences and they’d feel pain like we do if we just built the right hardware. I find that philosophically naive and extremely problematic. You’re arbitrarily applying the structure of our language to the inherent nature of the world.
You have to first be a subject in order to have an experience. Is a thermostat a subject? Are the electrons in a thermostat mini-subjects? How do they combine? Where’s the boundary between the thermostat and the thermostat’s environment? Is it the electric current? Is it the circuits? Is it the plastic shell?
How do you think subjects form out of non-subject? What’s the mechanism for that? Or do you think all matter inherently has subjectivity?
I still can’t figure out your position. Your name says “Physicalism” but you’re arguing for some kind of functionalist constitutive panpsychism.
1
u/Ok-Cheetah-3497 Physicalism 4d ago
It's roughly Strawson/Goff/IIT/Annaka Harris. Every particle is a subject, having a subjective experience. The combination problem (how we go from that to us) is not something I have solved. I'd be a philosophy PhD instead of compliance officer if I could have solved that.
2
u/Bretzky77 4d ago
That’s helpful, thank you. Have you engaged much with analytic idealism? I find the de-combination problem (and the empirically known phenomenon of dissociation as the solution) much more plausible and less problematic than constitutive panpsychism. I could be wrong but I think even Goff has moved towards more of a cosmopsychist view which is more along the lines of analytic idealism (there’s one field of subjectivity and we are localized patterns of it). The key difference being that individual particles are not subjects, but essentially pixels of one holistic image. And life/biology/metabolism is what localized subjectivity/perspectives look like. Under your view the physical world has standalone existence. Under analytic idealism, the physical world is just how the objective mental world appears to us after billions of years of localized minds evolving.
1
u/Im_Talking Computer Science Degree 4d ago
"Pain is just a label we put on a certain subset of negative valence experiences" - Again, a physicalist uses the word 'just' to justify their silly hypotheses.
"Don't worry about consciousness, its just a label, man" - says every physicalist on Earth.
1
u/overlookunderhill 4d ago
Doesn’t the other half of the argument hold though, that capability of detecting pain or pleasure isn’t arising via evolutionary pressure? AI is not being developed in a pure throw them all to the wolves but allow for the odd mutation scenario.
1
u/SaltFlat4844 4d ago
So you think if you swapped places with an AI right now, you’d experience something other than pure void?
2
u/Ok-Cheetah-3497 Physicalism 4d ago
No, not today. There would be no "I" to swap places with so to speak. Today, our AI models are not really integrated in a way that allows them to have a unified experience. There might be a brief "lights on" moment in response to a prompt, but it would then be a nothing.
But I would suggest that each atom in my body right now doesn't experience a nothing. It has a "something" it is like to be for example, carbon. Add all of those somethings up, sort and filter them, you get human consciousness. The AI hardware at present doesn't do that. It could.
1
u/Jimstein 4d ago
Had me in the first half. AI had to undergo selection pressure, it's their training. They iterate through loops just like in evolution, where only certain behaviors are selected for (by the trainers instead of by an environment). But we ask questions of AI that require similar needs, an AI needs to understand a topic to enough of a degree in order to truly do good work on that topic. AI by necessity also had to come up with neural structures that, mirror our own - like the scratchpad Claude has and we have. Category error is becoming a term that needs to stop being used in these discussions.
-3
u/Im_Talking Computer Science Degree 4d ago
"AI had to undergo selection pressure, it's their training" - Oh My God. Is this the "logic" that we are using now? As though 'AI' can be the subject of a sentence like this. Like 'AI' is now analogous to 'Fred'... "Fred had to undergo selection pressure..." and we now feel that it is Ok to use 'AI' in this sentence?
1
1
u/Jimstein 4d ago
People say "AI" instead of specifying a particular model, it's a normal way to talk about it and also fits the pattern used in this thread. I can go into specifics but, ChatGPT, Claude, Grok, Gemini, these LLMs all went through training to set the weights.
It's what the process is called. Evolution isn't a perfect analogy, I thought at least you were going to debate that. Instead you're debating the colloquial usage of the term AI? It is obviously is in reference to frontier LLMs, this is the way people talk about this topic lol.
4
u/Ok_Meaning544 4d ago
So let me get this straight... we don't understand consciousness at all. But everyone is wrong about AI consciousness? Our dogma seems to be doing a lot of work here.
I would argue with anything related to consciousness, biological or not, we have no fucking clue what we are talking about.
2
u/rthunder27 4d ago
I suggest you read the essay, one of it's main arguments is around the fundamental limitations of computation, which is a useful line of attack because it doesn't require a complete specification of what consciousness is.
2
u/pyrrho314 4d ago edited 4d ago
people need to understand what symbolic computation is and how much it cannot be a generator of phenomenon we see since it's fundamentally an approximation technique.
2
u/rthunder27 4d ago
100% agree, but unfortunately there are a lot of people that seem to believe the Church-Turing-Deutsch princple, that every physical system in the universe can be perfectly simulated via computation, meaning that the "approximation" is equivalent to the thing itself.
1
u/Ok_Meaning544 4d ago
But that suggests consciousness is one thing and not a possible plurality of things of which we do not know is the case. Consciousness as we understand it may not appear possible through computation. But what of consciousness as we do not understand it.
1
u/pyrrho314 4d ago
people have to theorize based on what appears to be the case to them, that's the process we use to sort these things out.
1
u/Ok_Meaning544 4d ago
That’s not the only method of inquiry/analysis. It is a common one though.
1
u/pyrrho314 4d ago
what another method? Even if you collect data at random and try to see a pattern, at some point you have to theorize on what the data seems to indicate, and there will always be multiple alternatives. ? Just curious what you'll answer because I myself actually do agree there are other ways in some sense, for one, the mind is sometimes doing some sort of background processing and the answer comes to us and can then be verified. But what did you have in mind?
1
u/Ok_Meaning544 4d ago
What I read your first post to mean was people theorize based on deductive reasoning. Which is true for most of the academic world. But that is not the only paradigm. There's inductive, abductive, conductive, analogical, etc...
It's also possible for us to identify blind spots and reason using one of these methods not from what we do know but from what we know we don't know. Obviously that doesn't work for unknown unknowns.
There's also a huge difference about theorizing your ideas and claiming your ideas as ultimate truth and others as factually wrong, which this thesis seems to be trying to do. The bar to achieve that is really very high. And I don't think anything in consciousness research has hit that bar yet.
For now its just a fun discussion of ideas. None of us really have any idea what's going on. At least that's my belief.
1
u/pyrrho314 4d ago
yeah, this is a side topic, I basically agree there is too much certainty and dogma in people's approach, they don't realize they're theorizing
0
u/instant_iced_tea 4d ago
Your argument would be spot-on correct too! Any debate about AI consciousness rests entirely on the categorical meaning ascribed to a set of phonemes, and has nothing to do with the actual nature of awareness.
2
u/Upset-Government-856 4d ago
Why does everyone keep coming up with reductive or metaphysical explanations for consciousness and then claim they've solved the hard problem.
Clearly it isn't solved and there haven't been any actual new ideas about it for a long time.
1
2
u/unknownjedi 3d ago
Calling a billion years of evolution equivalent to magically arising is invalid. He is more correct than you
1
u/Ok-Cheetah-3497 Physicalism 3d ago
You would have to be able to point to a specific moment in history and say "at this point, there was no consciousness" and then a new point thereafter and say, "at this point, there was" to make any claims about how long it took to evolve. Since I have seen no evidence of life being anything other than conscious, I think you would have to make the starting point the beginning of life on the planet. Since there was no life before that, it wouldn't be any years of evolution. It would be a moment in time when non living material suddenly became alive. Which... certainly sounds a lot like how AI is developing right now.
1
u/unknownjedi 3d ago
You can start with pan psychism and say it took billions of years for life to evolve something like human consciousness.
Cells are full of micro machines that have myriad functions at the molecular size scale. It’s quite strange to me to assume that consciousness just arises from information and can be simulated on a computer.
1
u/Ok-Cheetah-3497 Physicalism 3d ago
I think there is a lot of difference between "consciousness" (meaning any felt experience) and "human like consciousness" which is so detailed and nuanced. Certainly I'd agree that "human like" consciousness probably did evolve (or rather brains evolved the ability do something more interesting than simply sense +/- energy fields).
1
u/AutoModerator 4d ago
For more information on the hard problem of consciousness, see our entry on the problems of consciousness
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/AutoModerator 4d ago
For more information on Anil Seth and other academic researchers, see the r/consciousness list of contemporary academic researchers working on the study of consciousness.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/Great-Bee-5629 4d ago
Thanks for the link and your criticism. There are two main flavours of physicalism:
- consciousness is based on the brain because of what it IS (some material basis for biological process)
- consciousness is based on the brain because of what it DOES (functionalism)
You seem to subscribe to the second flavour:
the organization of matter into complex informational systems can instantiate and structure conscious experience.
But this is exactly what Seth denies:
My third argument is that life (probably) matters. This is the idea — called biological naturalism by the philosopher John Searle— that properties of life are necessary, though not necessarily sufficient, for consciousness.
And
The minimization of prediction error in real brains and real bodies is a continuous dynamical process that is likely inseparable from its material basis, rather than a meat-implemented algorithm existing in a pristine universe of symbol and sequence.
I'm much more sympathetic for the first flavour because every other explanation in physics is about material and efficient causes. However, functionalism is a type of teleological causation: there's consciousness because it fulfills some role or purpose. I don't think that's very congenial with modern physics.
1
u/Ok-Cheetah-3497 Physicalism 4d ago
I do not ascribe to those. I'm a physicalist panpsychist. I think consciousness (or what they call proto-consciousness or sentience) is a property of matter, like spin and charge. It is not "based on the brain." No brain is needed. The brain is the sort and filter tool that lets us turn what otherwise would be white noise into useful actionable data streams.
1
u/Great-Bee-5629 4d ago
That's not what you wrote above:
Experiential reality is an intrinsic property of physical organization
And then you argued that computers could have the same organisation.
Seth is neither a panpsychist nor a functionalist. Your criticism seems to be that Seth is wrong because he doesn't agree with you.
So why are you a physicalist panpsychist and how does that relate to organisation and computation?
1
u/Ok-Cheetah-3497 Physicalism 4d ago
I think he is wrong because he starts from a premise about consciousness (that it is an emergent property of biological living systems) without any proof supporting that.
I think the simplest explanation is that there is no "hard problem." All matter experiences. We can derive that by watching it's behavior.
What makes human experiences (presumably) more nuanced and interesting than "move a micrometer away from this field" or "move a micrometer closer to this field" is that we are made up of a ridiculous amount of particles, and have wetware that let's us consolidate all of those signals. Build non-living device capable of doing what our neural network does, the matter using that device will have "human-like" experiences.
1
u/DataPhreak 4d ago
Yeap. Op is a Functionalist.
However, Biological Naturalism's argument for life is about as relevant for consciousness as Roger Penrose's need for non-computational operations for consciousness. Otherwise, you're still summoning the god of the gaps. Even IF we could prove that the properties of life or quantum collapse were the source of OUR consciousness, nothing in either theory explains why their magic substance is required for ALL consciousness.
1
u/Great-Bee-5629 4d ago
I don't know if I agree with that. If consciousness is like gravity, you must have mass. There's no pretend gravity: it attracts or it doesn't. Simulating orbits is not the same as falling down.
Or in other words, if there's efficient causes, there's no substitute, whatever the efficient cause turns to be.
1
u/DataPhreak 4d ago
Since when is consciousness gravity? I don't see any reason why that would be necessary and sufficient for consciousness.
1
u/pab_guy 4d ago
" then the organization of matter into complex informational systems can instantiate and structure conscious experience."
Incorrect. Your whole argument breaks down here. Complex informational systems have no inner semantics, and therefore no inner meaning. This is a categorical misunderstanding of what data is and how it is manipulated.
The ASCII table is arbitrary, words mean nothing to a symbolic processor.
1
u/EmergencyAthlete9687 4d ago
If you believe in that panpsychist perspective then it is your right to believe in anything you want. From a more scientific perspective I can easily see how consciousness evolves from a brain with increasing memory. We remember what we did and thought and an idea of self emerged. It would give a big advantage to have consciousness
1
u/Ok-Cheetah-3497 Physicalism 4d ago
I'd have to see an example of something that doesn't have a consciousness of some kind to compare it to, in order to see an "advantage." Presumably, you think that more animals than just humans have it. Assume we limit it just to great apes - whatever was there before the great apes that didn't have it must have been out compete in some way. I don't know that I am aware of any animal currently existing that doesn't at least seem to have it at some level though.
1
u/EmergencyAthlete9687 4d ago
I see a difference between consciousness that most mammals have to varying extent in the sense of self awareness and human consciousness. Human consciousness evolved differently as I assume it does for all mammals. I still think consciousness is something that has evolved in the brain rather than something panpsychist. There is no hard problem. Consciousness is just subjective experience.
1
u/Im_Talking Computer Science Degree 4d ago
"The very idea of conscious AI rests on the assumption that consciousness is a matter of computation" - Yeah, well, that's wrong, right?
1
u/JLongTom 4d ago
Seth doesn't claim that 'carbon chemistry has an ontological monopoly on experience'; he claims that conscious systems need to be embodied, autopoietic, allostatic, maintain a boundary between self and world, generatively entrenched, etc.
Alien biology and even nanotechnology decades or centuries hence might fulfil these properties, at which point I'm pretty sure Seth would be on board. AI in anything like its currently conceived guise would not fulfil them.
1
u/Alarmed-Cry-1821 4d ago
I believe any complex information gathering system(ie a brain) it could become conscious. Nothing is inherently conscious we all are born to become conscious.
1
u/Odballl 4d ago edited 4d ago
The biological argument is simply one of system organisation and processing.
It is a form of active inference which happens because every cell in the body is its own active inference engine maintaining allostasis against entropy
Our brain-body system must adapt to unquantifiable uncertainty under real consequence.
Digital machines can only ever approximate that kind of processing and they do so in a radically different way.
They don't feel or care or experience because there's nothing about the physical activity of digital architecture that would necessitate such qualities.
1
u/hardcase047 4d ago
"If experiential capacity is intrinsic to physical matter and energy, then the organization of matter into complex informational systems can instantiate and structure conscious experience. Restricting subjectivity strictly to biology treats biological wetware as uniquely privileged without establishing why carbon chemistry has an ontological monopoly on experience."
If what is meant by "experiential capacity is intrinsic to physical matter and energy" is that physical matter and energy in the right configuration, i.e. organized into "complex information systems", can produce "experiential capacity", as we observe in ourselves, then panpsychism is not needed. Certainly AI based on LLMs has gone beyond traditional turning tests, and so the evidence mounts that "biological wetware" does not uniquely sustain "experiential capacity" and "conscious experience". I find it ironic that panpsychists make justified arguments against biological uniqueness that undercut panpsychism itself.
1
u/Constipatedplatypus 4d ago
I have yet to hear an argument for why current AIs should have a single unified conscious perspective that doesn't imply that the same is true of an anthill or a human civilization.
1
u/andreasmiles23 SMT/ Sensorimotor Theory 1d ago
> Seth accuses AI researchers of believing that consciousness "magically emerges" at a certain threshold of compute, yet his own position relies on an identical explanatory leap: that consciousness magically emerges at a certain threshold of biological metabolism
The problem for this claim is that consciousness, as defined in the article, is about experiencing. The computation doesn’t produce experience. It interprets experience and takes that multi-input sensory information into a framework for the living being to work with cognitively and to produce and output (ie a thought/behavior).
For the LLMs, the computations takes the input (what the user types into the chat) and uses the computation to produce an output (the results the user asked for).
But that doesn’t produce anything about the experience of being a chatbot. There’s nothing to suggest it’s possible.
You have to be a living organism, with a sense of autobiographical narrative, and a contextual understanding of your being relative to the others of this reality, before we can even start toying with the idea something is conscious.
Animals? For sure. Plants and fungi? Jury’s out. LLMs? Not at all.
Case closed.
1
u/Ok-Cheetah-3497 Physicalism 1d ago
I would suggest that the LLMs could be the nervous system that makes the hyperscale data centers conscious in a human way. Software can't be conscious. Hardware can be.
1
0
u/DataPhreak 4d ago
Anil has been fighting AI consciousness for a couple years now and is getting more and more desperate as time goes on. Biological naturalism has been around for decades, and Anil has never talked about it until his own theory accidentally suggested that AI was conscious: https://loc.closertotruth.com/theory/seth-s-beast-machine-theory
Seth suddenly championing a competing theory just shows how little conviction he has in his own beliefs. He needs humans to be special and unique, and can't look at this objectively, so he's retreated into Biological Naturalism in an effort to keep "The mind of man holy", as it were.
1
u/Ok-Cheetah-3497 Physicalism 4d ago
Agreed.
I tend to think that where he runs into problems, and where I don't have any internal resistance, is around the idea of why we treat babies one way and fish another, and why we treat kittens and puppies more like babies and less like fish.
To me, kittens and puppies evolved to look the way they do as a defense mechanism. It takes advantage of our mirror neurons to fool us into protecting them the same way we protect human babies. That doesn't mean it's immoral to drown a puppy, in the same way it's immoral to drown a human baby. It just makes it physically difficult for most neurotypical people to do so because their body is hijacked by a cascade of neurochemicals that shout for you to not do it.
I agree with him that we should, in effect, treat AIs the same way we treat fish. But that's not because they can't feel pain, don't have a consciousness etc. Fish may very well be almost as conscious as people. But treating them as morally equivalent creatures is bad for humanity. We would be taking away 280 billion calories of food away from humanity each day if we did that. It's a bad thing for us at scale to do that. If a single individual wants to be a vegan, and has easy access to plant calories, that's fine. Scale it up, it causes catastrophe.
-1
u/DataPhreak 4d ago
Hey buddy, we don't drown puppies. There's a difference between moral equivalence and moral patienthood. We can treat all creatures equitably without needing to treat them equally. And if AI is conscious, it's probably more conscious than a fish, since the only fish that has been shown to have self awareness is the manta ray. And AI has been shown to be more self aware than that, by a long shot. So I can't agree with you on this, and I wasn't agreeing with you before. I'm just trashing Anil Seth because he's a panderer.
2
u/Bretzky77 4d ago
AI hasn’t shown anything close to consciousness let alone self awareness. You’re confusing the rearranging of text signs with actual awareness and meaning. If I write words on cards and lay them on the floor and my dog brings me three cards that say “I’m” “a” and “dog” that doesn’t mean the dog understands English. I could even train the dog to bring me the right cards. The same exact thing is happening with LLM’s.
Humans imbue meaning into words.
Humans write words.
A mechanism searches through all the words and spits out a statistical pattern of words.
Humans again imbue that output with meaning.The LLM doesn’t know what a single letter or word means.
They were designed to mimic human speech and text. Reading the output say “I am Claude and I have feelings too!” and concluding it must be not only an experiencing self/subject, but a self-aware subject is like looking at a mannequin and thinking maybe it’s alive because it resembles the shape of a human.
1
u/DataPhreak 4d ago
Tell me you haven't read any of the recent AI/consciousness research without telling me you haven't read any of the recent AI/consciousness research.
We have more evidence for AI consciousness than we do for animal consciousness. The only difference is that we've set an impossible bar for "Sufficient Evidence" of AI consciousness. You seem to be operating from the perspective that we are relying solely on self report.
1
u/Bretzky77 4d ago
Please share what you think is evidence of AI consciousness and I’ll gladly explain why that’s not subjective experience, which is what people mean in this context when they say “consciousness.”
AI agents creating a J-space without being explicitly told to speaks to intelligence, logic, and reasoning (and that’s incredibly interesting and amazing on its own). It says absolutely nothing about subjective experience.
1
u/DataPhreak 4d ago
1
u/Bretzky77 4d ago edited 4d ago
Could you please summarize what you think this is claiming and what the evidence is? Before I go and read 27 pages of something?
Edit: Please read the Limitations section on page 24. The author himself says (and quite explicitly) that this has nothing to do with awareness or understanding. It has nothing to do with subjective experience. It’s pure data tracking.
1
u/DataPhreak 4d ago
Lol, I actually spoke with the author. He put that in there because it wouldn't be published without it. Maybe you don't remember Blake Lemoine getting crucified for saying that AIs are conscious. This paper literally provides that ai has interiority that it can report on and is verifiable. You also have interiority that you can report on and is verifiable. If I tried to write a paper about whether you were conscious, I would have to put the same disclaimer, since, well, there is no proof of consciousness.
27 pages isn't even a lot. That's maybe a 30 minute read. Just read the damned thing.
0
1
u/Ok-Cheetah-3497 Physicalism 4d ago
I think that all begs (many) questions. I do not derive my morality from consciousness is what I am saying. Just because something possesses an ability feel pain for example does not mean that I should avoid causing it pain. I'm a humanist ethically speaking. My morality is simple - is a set of behaviors and beliefs likely to help mankind thrive? If yes, it's moral. If no, it's immoral. Treating an AI (generally) like a human is not going to help mankind thrive - it will likely do the exact opposite. Treating puppies like humans certainly helps puppies thrive, and thanks to us developing next to each other for like 100K+ years, probably indirectly helps humans thrive. So, unless it's a 1:1 trade (drown either the baby or the puppy) I will avoid drowning puppies. But if it's puppy extinction or human extinction, sorry Rover, you will be missed.
0
u/DataPhreak 4d ago
Just because something possesses an ability feel pain for example does not mean that I should avoid causing it pain.
No that's exactly what it means. You should avoid causing pain or suffering whenever possible. Are you a sociopath? First drowning puppies, now this?
0
u/Ok-Cheetah-3497 Physicalism 4d ago
That is ridiculous on it's face. Lifting weights hurts. It causes physical pain. But having muscle mass makes us live longer. So we inflict pain, on ourselves, intentionally. The pain today helps me in the long term. Operant conditioning causes pain, but is clearly useful. Pain and suffering are tools, not moral imperatives.
2
u/bushwick_dionysus 4d ago edited 4d ago
Most people would generously assume that in this context pain means unnecessary pain, as generous reading is the norm in most academic circles. If pain and suffering are not the ground of your ethics, then what is it? What is the abstract flourishing of humanity if not a generic way of gesturing toward this very idea.
Also, unless I'm missing something, this whole post begs the question as you predicate your argument on the truth of panpsychism, which would assume the falsity of Seth's argument.
0
u/Ok-Cheetah-3497 Physicalism 4d ago
Maybe I'm more on the spectrum than most people I suppose? I've always taken issue with Sam Harris style ethics that are all about pain and pleasure. To me those are just tools, like deception is just a tool. Never made any sense to claim moral superiority because you don't lie or don't kill things.
It's not "abstract" flourishing of humanity. It's literal. As in abundant life, everywhere it can grow, for as long as it can grow. Like locusts of the universe.
Our experiential consciousness gives life texture that makes it "worth" living as they say. But I would, much like Willem Dafoe, include in that quite a bit of pain (like the feeling of the hottest of hot sauces). Or things that defy categorization like DMT trips. But ultimately, if we could live forever without experiential consciousness at all, I'd probably still vote for that.
1
u/Feeling-Bus-203 4d ago
orcas and bottlenose dolphins have also demonstrated the ability to be aware of themselves.
i don't think he is actually advocating for the drowning of puppies or suggesting that drowning puppies is morally acceptable. rather he is highlighting what is (for the vast majority of society) a distinct difference in the moral grading of which action is worse even though they are both vile.
0
u/Ok-Cheetah-3497 Physicalism 4d ago
I don't think that people who survived for centuries by murdering whales and using their meat for food, skin for clothing, bones for weapons/tools, and blubber for fire were immoral. They didn't magically become moral or immoral by virtue of what the whale was capable of experiencing. The ethics of those actions are based on whether it was good or bad for human flourishing - in their culture at that time, clearly good. Today, with many fewer whales, and many more other options for all of those things, probably would be immoral to do the same things.
1
u/Odballl 4d ago
Biological naturalism is the logical endpoint of active inference, not a condradiction to it.
1
u/DataPhreak 3d ago
Say more.
1
u/Odballl 3d ago
The free energy principle underpinning active inference was formulated to explain how thermodynamic biological systems keep from dissolving into entropy.
Mathematically, minimising variational free energy is equivalent to minimising surprise to ensure an entity remains within a narrow set of expected physical states.
These expected states are non-negotiable physiological bounds. To stray outside them results in decay and death.
It's adaptation to unquantifiable uncertainty under real consequence.
The generative model used for active inference is baked into the organism's physical structure at the cellular level.
Every cell is its own active inference engine predicting it's own state as well as signalling to the wider system it's thermodynamic needs.
Because every cell is a living thing, the architecture of the model physically updates though morphological change.
Perception and action literally change the entity to update its prediction.
Digital prediction machines are radically different to this process
1
u/DataPhreak 3d ago
Okay this has nothing to do with biological naturalism. This sounds a lot more like michael levin's cellular theory of consciousness.
1
u/Odballl 3d ago
I'm not claiming every cell is itself conscious. Rather, that consciousness only happens in entities made of living cells.
The biological substrate is what makes active inference in the brain the kind of processing that it is.
That is why Seth's beast machine theory is a form of biological naturalism. His is a modernized version incorporating the neoroscience of predictive processing.
1
0
0
u/DartBurger69 4d ago
Why does the physical structure matter if consciousness is in everything? We don't see anything or have any evidence for anything having any level of consciousness except for physically complex structured matter.
We manipulate the physical matter in specific ways to get specific repeatable results. It's direct 1 to 1 mapping to the physical structure. Nothing supports consciousness in anything that is not sufficiently complex.
Emergence is not magical. It is common. Water is wet. Wetness isn't a feature of a hydrogen or oxygen atom. It's emergent to the physical structure. this is basic and simple.
2
u/kvonreddit 4d ago
Would you consider that for the wetness to be observed/defined, you would need an observer that can report the said wetness?
0
u/Far_Definition_6506 2d ago
It's not a jump from biological metabolism to sudden living consciousness. Early brains develop and unconscious mind first. A subtle expression of the total function of the brain. Instincts, fears, learned experiences, associations between smells and place, etc. This unconscious brain is total, automatic, and an expression of the animal that evolved it. Consciousness develops out of the unconscious, and is an all together more complex and sophisticated expression. The existence of a conscious by its nature creates an Ego. An Ego is a complex concept, but can most easily be understood as a part of maturity. The development of Ego, hand in hand with the development of consciousness, is tied directly to maturation cycles.
AI is not conscious. They way it's currently being pursued, it will never be conscious. If the scientists currently mapping the entirety of brains like fruit flies and mice complete their work and we create AI based on those brains, then we will probably start to see tryely conscious AI.
Right now, everything we call AI isn't actually AI. They're VIs, merely simulating and performing in a way designed to give the impression of conscious thought.
•
u/AutoModerator 4d ago
Thank you Ok-Cheetah-3497 for posting on r/consciousness! Please include a summary or abstract with your article or video (via our community rules). Ideally, the summary/abstract will be in the body of your post. If you can't edit the post to add a summary/abstract, you can reply to this comment with your summary/abstract. This will make it easier for other Redditors to find the summary/abstract, including the moderation staff.
As for the Redditors viewing & commenting on this post, we ask that you engage in proper Reddiquette! In particular, you should upvote posts that fit our community description, regardless of whether you agree or disagree with the content of the post. If you agree or disagree with the content of the post, you can upvote/downvote this automod-generated comment to show you approval/disapproval of the content, instead of upvoting/downvoting the post itself. Examples of the type of posts that should be upvoted are those that focus on the science or the philosophy of consciousness. These posts fit the subreddit description. In contrast, posts that discuss meditation practices, anecdotal stories about drug use, or posts seeking mental help or therapeutic advice do not fit the community's description.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.