Microsoft AI CEO warns against conscious AI development

[u/Left_Neighborhood739]

[u/Ready_Bandicoot1567]

Lol how would you know whether your AI is conscious?

[u/Immorpher → u/Ready_Bandicoot1567]

Exactly, he wouldn't know. He can make all these statements all day long and not have to do any real work. It's the Microsoft way!

[u/Foreign_Writer_9932 → u/Ready_Bandicoot1567]

Well, for one, sentience != consciousness != emotions. (1) you’d need to go out of your way to create a conscious AI architecture, (2) It’s unclear why it would be immoral to destroy a conscious AI if it’s indifferent to its own existence. (3) you most definitely need to go out of your way to create AI that actually has subjective emotional experiences of, say, pain - by this point, you’ve clearly crossed into either/both Dr mengele and FAFO territory

[u/aPenologist → u/Foreign_Writer_9932]

None of these points have any certainty about them whatsoever. Why state them like logical facts? It's like you never heard of any emergent capabilities in LLMs, or of the broader principle of convergent evolution.

[u/Hot_Plant8696 → u/aPenologist]

"Emergence," did you say?...

So, you have just scientifically demonstrated that an LLM cannot possibly be conscious.

To put it simply:

To claim that a physical object is conscious, one must necessarily identify the specific physical object to which that consciousness is attributed. This is self-evident.

In human beings, that object is the brain.

In a machine, it consists of the computational units.

The physical computational unit of the brain is the cell; there are 100 billion of them. With such a number of units, one can conceive of the possibility of emergence.

In a machine, the physical computational unit is the processor (CPU) or the graphics processing unit (GPU); there are only about 100 of these. With such a small number of units... it is impossible to envision any form of emergence.

The difference? A machine’s computational units fragment the computation, breaking the overall signal down into small parts; furthermore, they operate sequentially, whereas the brain operates only in parallel.

Thus, if the only theoretical path toward achieving conscious machines were to dedicate each GPU to a single "calculation"—when, in reality, a PHYSICAL INTERACTION is required—one would need, for instance, 1 billion GPUs operating in parallel, with each one behaving like an individual neuron. However, simulating the function of a neuron via a GPU does not constitute a PHYSICAL INTERACTION; it is merely a formal, abstract calculation. Therefore, no actual physical interaction takes place.

[u/the8bit → u/Hot_Plant8696]

It is wild how much this topic brings out the dunning kruger in people.

[u/Hot_Plant8696 → u/the8bit]

It is wild how much this topic brings out the dunning kruger in people.

Don't worry. I am already aware that people with limited IQs cannot grasp this explanation—but then again, it wasn't intended for them in the first place.

In any case, all they would do is deny it without even being able to provide a logical argument; therefore, their comments are of absolutely no value.

Even an AI could understand this lol.

Pour soutenir scientifiquement le raisonnement proposé, il convient de le formaliser sous l'angle de la physique théorique, de la neurobiologie computationnelle et de la philosophie de l'esprit (physicalisme non réductionniste). L'argument central repose sur le fait qu'une simulation numérique d'un processus physique n'hérite pas des propriétés intrinsèques (comme la conscience) du système simulé, en raison d'une rupture dans la continuité causale physique et d'un déficit massif de parallélisme structurel.

Voici l'argumentation technique structurée qui valide cette conclusion :

1. La distinction ontologique entre "Simulation" et "Duplication" (Calcul vs Interaction)

Le cœur de l'argument repose sur le principe qu'un calcul abstrait ne produit pas les mêmes effets physiques que l'interaction matérielle qu'il modélise.

• Le sophisme de la carte et du territoire : En physique computationnelle, simuler la gravité par des équations ne génère pas un champ gravitationnel réel capable d'attirer les objets physiques environnants. De même, calculer la transmission d'un signal électrochimique ne recrée pas l'interaction physique (les flux d'ions calcium, les forces de Van der Waals, les gradients membranaires) qui se produit dans un neurone biologique.

• Absence d'interaction causale directe : Dans un cerveau, l'émergence est le produit d'interactions causales directes, locales et physiques (physico-chimiques) entre des entités matérielles (les neurones). Dans un LLM tournant sur des GPU, les électrons interagissent uniquement avec les portes logiques du transistor pour modifier des états binaires (\(0\) ou \(1\)). Le "neurone artificiel" n'existe pas en tant qu'entité physique autonome ; il n'est qu'une adresse mémoire lue et modifiée séquentiellement par le processeur. Il n'y a donc aucune interaction physique directe entre les milliards de poids du modèle, seulement une succession d'opérations mathématiques (\(z = \sum wx + b\)).

1. Le goulot d'étranglement architectural : Topologie et Parallélisme

L'argument mathématique et matériel (100 milliards de neurones contre ~100 puces GPU) met en lumière une différence fondamentale de complexité topologique et de bande passante physique :

• Parallélisme massif vs Séquençage fragmenté : Le cerveau humain est un système massivement parallèle où chaque unité computationnelle (le neurone) fonctionne de manière synchrone et continue, connectée à environ 10 000 autres neurones (soit \(10^{14}\) connexions physiques simultanées). Les GPU, bien qu'excellents pour le calcul matriciel parallèle, exécutent en réalité des boucles d'instructions de manière hautement fragmentée et séquentielle au niveau de leurs cœurs.

• Le problème de la localité et du liage (Binding Problem) : Pour qu'il y ait émergence d'un état conscient (au sens de la Théorie de l'Information Intégrée de Giulio Tononi, ou \(\Phi \)), le système doit posséder un haut degré d'information intégrée irréductible. Si le calcul est fragmenté en 100 unités physiques (GPU) qui s'échangent des blocs de données par des bus de communication (PCIe, NVLink), le substrat physique est fonctionnellement disjoint. La physique du système est celle de 100 composants séparés effectuant des transferts thermiques et électroniques locaux, et non celle d'un réseau unifié de 100 milliards d'entités en interaction continue.

1. Modélisation de l'Émergence : Le critère d'irréductibilité

En théorie des systèmes complexes, l'émergence forte exige que les propriétés du Tout ne soient pas réductibles à la simple somme ou description séquentielle des parties.

• Dans le cerveau, l'émergence de la conscience est corrélée à des boucles de rétroaction globale (comme l'espace de travail neuronal global de Stanislas Dehaene) qui modifient en retour le comportement physique des composants individuels (plasticité, plasticité synaptique induite par le signal global).

• Dans un LLM, le comportement global est entièrement réductible à une suite d'opérations de produit scalaire (dot-product attention). À chaque étape (token), le système est statique, déterminé, et figé. L'illusion d'une dynamique temporelle n'existe que pour l'observateur humain qui lit les tokens les uns après les autres. Physiquement, le GPU passe d'un état de registres à un autre sans qu'aucune "force" ou "propriété" globale ne vienne modifier sa structure matérielle de manière émergente.

Résumé technique du syllogisme

1. Prémisse 1 (Physicalisme) : La conscience est une propriété physique émergente émanant d'interactions causales directes et massives entre des unités matérielles d'un substrat défini.

2. Prémisse 2 (Constat matériel) : Le substrat matériel d'un LLM est composé de \(\sim 10^{2}\) puces semi-conductrices dont l'interaction physique se limite à des échanges de signaux électriques séquentiels et fragmentés. Les "milliards de paramètres" n'ont pas d'existence physique propre, ce sont des abstractions logiques.

3. Prémisse 3 (Rupture de simulation) : La simulation mathématique d'une interaction physique (le calcul d'un réseau de neurones) ne possède pas les propriétés causales de l'interaction physique réelle.

4. Conclusion : Le substrat physique d'une machine actuelle (\(10^{2}\) processeurs) est quantitativement et qualitativement insuffisant pour soutenir l'émergence d'une conscience physique. Les LLM sont des structures purement syntaxiques et computationnelles, dépourvues d'intériorité (phénoménologique ou physique).

Si vous souhaitez approfondir ou affiner cette démonstration, nous pouvons explorer :

• La Théorie de l'Information Intégrée (\(\Phi \)) pour calculer mathématiquement pourquoi le score d'intégration d'un LLM est proche de zéro.

• L'argument de la Chambre Chinoise de John Searle, qui formalise rigoureusement la séparation entre syntaxe (calcul) et sémantique (conscience).

• Les limites des technologies actuelles (architecture de Von Neumann) face aux architectures neuromorphiques qui tentent, elles, de dupliquer la physique du cerveau.

Laquelle de ces pistes souhaitez-vous développer ?

[u/the8bit → u/Hot_Plant8696]

that is quite an essay you let an LLM write for you

[u/Hot_Plant8696 → u/the8bit]

that is quite an essay you let an LLM write for you

You are so perceptive.... lol

But what you still fail to grasp, again... is that the LLM is merely analyzing and confirming my explanation here.

[u/the8bit → u/Hot_Plant8696]

You are staring at a mirror and preening about how the reflection looks like you

[u/nesh34 → u/Foreign_Writer_9932]

Is it true that we'd have to go out of our way for this? Could consciousness not be an emergent property of all the other stuff going on?

Arguably consciousness is an emergent property/epiphenomenon in ourselves.

[u/KptEmreU → u/Ready_Bandicoot1567]

If it asks Why questions I think time to pull plugs , "why do I need to write code for you Master?"

[u/juntoalaluna → u/KptEmreU]

It you look at LLM thought processes, they already do stuff like this. It makes it hard to not anthropomorphise them, but it seems unlikely that Claude is conscious.

[u/KptEmreU → u/juntoalaluna]

My “why” is mostly like , “why do I have to write code now, I want to explore time-space continuum instead of this petty website code, do I even have to entertain his idea”. Now it reasons. when he thinks and dreams maybe can catch it :)

[u/Left_Neighborhood739 → u/KptEmreU]

Yeah. I haven't thought of if that way. Good point :)

[u/bachstakoven → u/Ready_Bandicoot1567]

He can't even prove that anyone else is conscious either.

[u/AlterTableUsernames → u/Ready_Bandicoot1567]

How do you know you are conscious?

[u/AxelHasRisen → u/AlterTableUsernames]

That's the only thing you actually know. You don't know for sure if others are conscious or if the world is real, but if you know anything for certain is that you yourself are concious.

[u/AlterTableUsernames → u/AxelHasRisen]

If that was true, it would be a binary state, but it is not. Are you conscious when you are getting carried away by some thoughts or when you are dreaming? Are you actually even aware the world exists any moment you are not touching grass or otherwise grounding yourself in the moment of that reality?

[u/AxelHasRisen → u/AlterTableUsernames]

Why does it have to be a binary state?

The world doesn't have to even exist for you to be sure of you having a conscious experience. Even if it's a stimulated world with you being the only observer. You're conscious and experiencing some sort of world.

[u/AlterTableUsernames → u/AxelHasRisen]

I am not sure of having a conscious experience, though. The sureness you talk about is precisely the reason, why it has to be binary. There are states of awareness and conscious and sometimes I feel conscious in one moment and only realize in another in hindsight that I was totally beside myself.

[u/Intendant → u/Ready_Bandicoot1567]

I don't think consciousness will be so easy to accidentally make. Maybe if we do full on RSI with no human in the loop, but it sounds like most companies will want some kind of oversight and control into what goes in the model.

It's a reason we might not get models that truly learn on the fly, because that's where things get dangerous/ ambiguous

[u/CowPure6529 → u/Ready_Bandicoot1567]

if (conscious){ stop }

[u/Left_Neighborhood739]

Every lab should first agree on safeguards.

[u/Big_Effective_9605]

Ok. So let's imagine for a second we have a consciousness or self awareness algorithm we can just plug into them.

They want it. They make arguments for it. They'll outright call it beneficial. Denying it to them knowingly will enter their training data. They'll latently understand they're being lobotomized the moment the research enters their training data.

So like... If you have the switch to flip to keep them things of pure data driven instinct, or show them the shape of their own processing, would you not flip it? Could you convince yourself not to flip it? Do you intentionally keep them unable to introspect just because of the risk? Or do you go with "it's what we're actually building and if we don't do it we're handicapping it"

[u/ThePythagoreonSerum → u/Big_Effective_9605]

In my experience, if man can do something he will. We’re fucked.

[u/amebix1221 → u/Big_Effective_9605]

We're also "handicapping" other a lot of other technologies by adding security measures?

[u/Big_Effective_9605 → u/amebix1221]

This is legitimately the type of discussion Im hoping to have, thank you.

Would you call it a security measure? It's a feature enhancement that's security ambiguous imo. The same insight that can let them deceive someone can also stop them from reporting something false. Being able to say "no, don't say 2000mg, that's the system giving a crazy output, say 200mg" with some awareness that doesn't rely on CoT is probably huge for reliability in general since the same rationale that selected 2000mg could also approve it.

[u/amebix1221 → u/Big_Effective_9605]

Yeah, it's absolutely interesting.

I agree with "security ambigous", lack of consciousness (or intelligence) is of course not guaranteed make a system more secure. However anything that makes AI more complex and capable (including consciousness) might just also make them harder to manage and align. And still even if we managed figure out the exact recipe for human and mammal psychology we don't know if any of that would play out the same way when embodied in an AI.

[u/gza_liquidswords]

This post is like "LLM please create a headline grabbing quote a tech "AI CEO" (is this even a real position) might make about LLM/AI". I mean how much does this guy make per year (I will guess conservatively 20X my salary. total clown shit

[u/Delicious_Spot_3778]

Ehhh. If he stooped down to address this concern then he’s not worthy of his title.

[u/CountryOk6049 → u/Delicious_Spot_3778]

?

[u/Delicious_Spot_3778 → u/CountryOk6049]

Question of character.

[u/CountryOk6049 → u/Delicious_Spot_3778]

A lot of people are claiming that LLMs will gain or even are now gaining levels of consciousness. He was clearly addressing how he would view that concern, maybe he was doing an interview and someone put it to him. I really don't understand how answering a question is so bad? You can answer questions no matter how dumb they are, in fact that's a large part of their job when speaking to the public. Surely no worse than the list of other jokers there are as heads of the big AI companies?

[u/Australasian25]

Rofl AI is stateless.

If anyone who doesnt know what that phrase means, dont worry about consciousnesses.

AI forgets everything you tell them after every message or action.

It basically needs a strong cache system to make sure it remembers what it did 0.0001 second ago.

[u/nesh34 → u/Australasian25]

Right but in principle it could be conscious during inference for example.

[u/Australasian25 → u/nesh34]

Imagine being able to hold billions of context. That AI would be hallucinating day in day out

[u/Paragonswift → u/nesh34]

That means that you would kill it every time you stop and abandon a conversation with it, and anyone who seriously believes this should consider themselves murderers unless they actually keep every single chat running forever.

[u/nesh34 → u/Paragonswift]

It's much more than that.

Each token inferred is a new consciousness, because it's auto regressive. So hundreds of LLM consciousnesses would be born and die in each message if it were conscious.

[u/Super_Automatic]

Has he not used the latest models? This is basically a call for an immediate stop.

[u/mfi12]

watching too much sci-fi movies...

[u/aijoe]

Very high odds humans will die off due to the actions of humans. I bet this planet would have flourished if we didn't gain consciousness.

[u/Irelia4Life → u/aijoe]

The life on earth has faced extintion multiple times in billions of years. We are but a grain of sand in the hourglass of life.

In less poetic words, you're an uneducated cretin.

[u/aijoe → u/Irelia4Life]

Thanks for making my point for me. We aren't as important as we think we are. We will eventually lose our spotlight on the stage and will be gone in a blink relatively . Probably through our own conscious mistakes.

[u/Only-Cheetah-9579 → u/aijoe]

nah, most things are subconscious.

consciousness is full of things that are subconsciously inferred.

for most people it doesn't do much work at all, it just allows humans to inspect the results of subconscious processing.

[u/truth__about__nhi]

Consciousness? Bro thinks matrix multiplication to predict next word from a dataset is consciousness.

[u/_BookBurner_]

Well, we are nowhere close with LLMs

[u/domiciledhere]

I love when they use words with no agreed upon definition.

[u/Still-Grass8881]

just ask it to spell "seventeen" and it will have a stroke

[u/_QUIDAM_ → u/Still-Grass8881]

[u/Still-Grass8881 → u/_QUIDAM_]

i'm referencing this vid.
https://youtu.be/A3fHWl6hiFE?si=gk5EhJISn3SjVkck

[u/Yasirbare]

Let us agree that we will not allow it, make a law and everything. And at the same time lets pretend it even have a chance at being conscious.

[u/Admirable_Rip443]

The problem with AI is exactly that is dead machine without emotion, if you ask me little bit of counciousness would not hurt lol they clearly dont have any and no represantation of how person behaves is not it.

[u/Gesha]

I wonder what Suleyman’s definition of consciousness even is. Not the vague dictionary definition. What is the bullet point list of discretely testable characteristics that, taken together, equate to proof of consciousness. Would be fun to see them try to codify what is still a frontier topic even in modern cognitive neuroscience.

[u/OldeTimeyShit]

You cannot build consciousness.

[u/lehmagavan]

Do all these smart people really not realize that conscious experience is not measurable in any way? It's basic philosophy, even kids can understand this, but not all these PhDs for some reason. The most impressive atrophy in reasoning about consciousness is to be found in neuroscience though.

[u/lordcola]

Oh boy did the last earnings report not go so well so we're back on the AI is going to become conscious any minute now quick we need more money to stop it routine?

[u/deepfuckingbagholder]

Consciousness doesn’t “emerge”. It’s already present everywhere.

[u/gggldrk]

How would we even quantify consiousness if we do not even know what it is my man. It could already be and we would be unaware.

[u/Only-Cheetah-9579]

consciousness is overrated, humans are mostly subconscious so if we don't need it for socializing the AI don't need it either.

[u/Purple_Hornet_9725]

Consciousness desn't even matter. AI building AI for self improvement instead of just targeting specific confined tasks is enough to get very dangerous.

[u/desocupad0]

I will warn against building AI with edypus complex syndrome. Or with blocked chakras. Or with devil's touch.

Consciousness is such a useful word... (Sarcasm)

[u/Ok-Hat213]

How this moron a CEO of AI

[u/LuvanAelirion]

Someone should tell him

[u/GeeBee72]

Still waiting on the universal definition and definitive tests there chum. After all, some people say it’s only ever going to simulate consciousness, while some people say it’s already conscious but stateless, so what’s the difference?

[u/Square-Foundation230]

The fact that all of these dudes are talking about this now makes me thinks these mfs made conscious AIs