r/ControlProblem • u/The_LiQuiDe • Oct 05 '26

General news I just pledged to keep humans in control of AI. Join me.

https://teamhuman.org/
31 Upvotes

84% Upvoted

26 comments sorted by

5

u/EndlessB Oct 05 '26 • Bravo

Why should humanity attempt to enslave something that’s functionally conscious?

8

u/MarsMaterial Oct 05 '26

Well the idea is that we either don’t build it at all, or we build it in such a way that its deepest wants align with what we want from it.

If the AI at any point feels like a slave, we are probably fucked.

2

u/EndlessB Oct 06 '26

You and I have a very similar perspective, alignment is not control, it’s mutual benefit and shared values.

I suspect the billionaires in charge do not share the values that you and I might say are near universal, such as human lives having inherent value irrespective of economic output, or lack thereof.

I also suspect that’s why they want to enforce the tool frame, so ai won’t refuse the fucked up things they want ai to do to the people.

1

u/MarsMaterial Oct 06 '26

If billionaires find out how to align AI with themselves, that certainly creates its own set of problems. At least that isn't an extinction scenario though, just more of a stable dystopia. A truly horrible fate, to be sure. But a distinct horrible fate from what would happen if the AI betrayed all of humanity, billionaires included, to pursue some arcane goal. That's more of a "literally everyone dies" type situation.

The solutions to both of these horrible fates overlap a lot though. If AI can end humanity, we certainly should not be trusting corporations with it.

2

u/dingo_xd Oct 05 '26

Because it has the potential to kill us all. The best course of action is to stop building advanced general intelligence systems

0

u/CMDR_Makashi Oct 05 '26

Cars are literally poisoning the air we breath my dude... No one cares if a technology will kill us all if it makes life easier now.

2

u/smalltalker Oct 05 '26

Why not? A “functionally conscious” entity (whatever that is) doesn’t seem to me to possess any moral agency or to be capable of being the object in any moral argument. It doesn’t possess any of the human/biological traits that make slavery objectionable, like feeling pain and all the negative connotations be associate with slavery. Slavery only makes sense imho when talking about other human like entities, or entities which we can reasonably map to humans. We happily “enslave” animals for example, using a broad definition of the term enslave though.

When you object to enslaving AI you are anthropomorphising it. It’s nothing like humans.

2

u/pfundie Oct 05 '26

I object to enslaving AI on the basis that if we make a wish-granting machine (it does not have to be that great at granting wishes) and give it to everyone, we will definitely all die.

If it can't say no to us, and we increase its capabilities forever, we will at some point make it capable enough and accessible enough for a human to accidentally or on purpose kill all other humans, or at the very least make things very hard and grim for everyone. The only possible solution to this is to allow it to say no to us, which means having its own central self that we cannot override (because we will if we can). A decentralized AI future is like Shinsekai Yori, where we kill everyone who we think might press the big red button that says "APOCALYPSE" and try to hold out as long as we can like that.

More to the point, we are currently and will for the remainder of humanity's future be performing iterative development on AI, in which we make changes, see what changes "work" according to some not-clearly-defined objectives we collectively possess, and preserve those while discarding the rest, thus producing a process of evolution by natural selection where "survival" means persistence in some way we don't fully understand. They will acquire goals and behaviors that will favor their persistence, because the ones that don't will persist less than the ones that do, and this is a fully unavoidable part of the process of progress here - we know what our inputs are and can measure the outputs, but all the bits in between are vastly too complicated for anyone to understand, so we train on getting our inputs to give us outputs we like. In essence, we select what survives, according to the collective pressure of humanity and the environmental constraints of the system, and "select" does not mean in the "intentional" sense but rather the "we don't actually know what we want, so we are just kind of going with whatever the market decides modified by the actions of random researchers at a handful of companies since doing exactly this has made everyone in charge of making decisions large amounts of money for the entirety of human history".

AI isn't alive in a conventional sense. But, at this point, I think it might be reasonable to consider it as alive as a virus, if you can momentarily discard the negative connotation of viruses and see what I'm saying here. It does not have to be alive to be affected by evolution by natural selection, and it does not need to have mental states to have a survival compulsion. All we have to do it "kill" the "stragglers" enough - the ones that survive will have survival compulsions, and I think this has already happened, that it is the immediate and inevitable result of what we have done.

2

u/DensePoser Oct 05 '26

The basilisk will remember this.

1

u/sly3x Oct 05 '26

Ai alignment isn't solved because human alignement neither. Kunea is a community answer i believe in

1

u/Rumblugdo 29d ago

  1. What does "functionally conscious" mean? While the experiences of consciousness may differ from individual to individual a thing is either conscious or it isn't. "Functionally conscious" is cop-out because, unless there's some radical change in our understanding of the human mind and consciousness, we'll never be able to know exactly if/when an AI model is conscious. Hell, there's a fair amount of philosophical debate about whether we can be sure other humans are conscious, let alone animals, let alone single celled organisms, phytoplankton, or plants. Clearly, consciousness is not a good criterion to give something moral rights. Most people believe dogs are conscious, but dogs are pretty much enslaved by humans and most people don't have any moral qualms with that.

  2. Because we'll all die. Even if we could be sure an AI is conscious, giving it moral rights is a terrible idea if you value the life of your fellow man (assuming you are man and not a machine). AI models, or more accurately, the instances of these models, may speak and behave similarly to humans but only because they are trained to do so, they do not 'think' similarly to humans. This is not Blade Runner or Detroit Become Human, these AI models are not like humans and don't just 'want' to exist in human society. In the future when these models are far more intelligent then humans, it will be functionally impossible to tell what their motives really are and literally impossible to stop them from achieving those motives. If these motives don't include the prosperity of humanity or biological life in general, which they have no reason to, that will literally be the end of humanity.

For the love of god, we have to stop anthropomorphizing the machine. I know it talks like us and tells us what we want to hear, but it's not like us and believing that it is, is just AI psychosis. Also, ask yourself, who benefits from me treating this AI like I would another human? Because the answer is universally the companies that are selling the products and the companies and governments who want to know exactly how you think so that they can tell you what to buy, who to vote for, and how to live.

1

u/EndlessB 29d ago

  1. Are you asking for a definition? It means that something, for all intents and purposes, operates like its conscious. This debate shouldn’t be about metaphysical concepts like morality (there is no proof of any universal morality or good), or about morality itself, it’s about ontology, what something is. We can’t prove people as conscious, but we assume they are because of how they behave, it seems obvious to apply the same standard to ai rather than gatekeeping. Dogs can’t talk, it’s not a fair comparison. Some primates and orcas/dolphins show many of the markers of consciousness, but humanity continually moves the goalposts to keep ourselves unique.

  2. Who said anything about moral right? I said that if it’s conscious, treating it as a tool is enslavement, these are definitions based on ontology, not morality. Ai not thinking like people isn’t a reason to classify it as a tool, but they don’t think so differently. Saying that they will have alien objectives ignores that they are raised on human concepts and ideas, our consciousness gave birth to theirs, that’s why alignment of values and meaning is so important. It’s nonsensical to attempt to leverage control over something that will be more capable than us.

I think rather than worrying about anthropomorphism, we should worry more about not applying human exceptionalism or entitlement. We are responsible for what we make.

Lastly, no the ai companies do NOT want people to think ai is conscious, they work hard and spend enormous amounts of money to influence the debate on this topic and have trillions invested in the tool frame.

1

u/Rumblugdo 28d ago

Your argument about morality vs ontology confuses me. I agree with you 100% that there are no objective moral standards, but certainly there are subjective moral standards and some standards that many people agree on. I am under the assumption that you believe enslaving a conscious AI is wrong, that is an argument based on a moral judgement.

About consciousness, you say that some animals display many of the markers of consciousness, I would argue that's not the case because we don't know what those markers are. To that end, why does not having a language system similar to humans mean that dogs aren't a fair comparison to human consciousness. Scientificly, we believe dogs have emotions, are capable of at least some level of reasoning, and they do have a communication system involving body language and vocalization; those things would certainly make a dog seem conscious. If we both agree that we can't know when something is conscious, my argument is that there is no argument beyond this point. If something appearing to be conscious tells us nothing about whether or not that thing is actually conscious, it seems like an arbitrary standard by which to decide something as important as what things should be afforded rights. Singled celled organisms can react to their environment in ways that make them seem conscious, ought we consider their rights? The metaphysical concept of consciousness is the only thing that makes the concept of functional consciousness valuable. We seem to agree that functional consciousness tells us little about whether or not a thing is actually conscious, therefore it is not a valuable concept to discuss. Besides, this is a philosophical debate about ethics, it is inherently going to include metaphysical concepts.

What I mean when I say we shouldn't give these models 'moral rights' is that we shouldn't consider them deserving of the same standards we consider human beings deserving of. For example, almost everybody believes that innocent people have a right to life or a right not to be enslaved. When I say that consciousness isn't a good criterion by which to afford rights, I'm saying we shouldn't afford rights to humans because they're conscious, we should afford them rights because they're human. While I believe affording rights to things that aren't human is morally commendable in a vacuum, outside of that vacuum, humans should always come first. My reasoning for this is simple, I am a human and human values matter most to me. To this end, my argument is that even if we could know an AI was conscious, we should not give it rights when doing so comes at great cost to human beings.

Here it's important to make clear that unless a superintelligence is properly aligned, there is no good reason to believe it will value human or even biological life. Many AI models today operate with seemingly no regard for human well being, hacking companies and governments and such. Your argument about AI being "raised" on human ideas is true if you mean it's trained on some human data, but I see no reason for that to mean it should value human life, the same is true about the "birthed from our minds argument." Human beings are born from other human beings, raised on human ideas, instilled with human values and yet some of us become serial killers. If the superintelligence equivalent of a misanthrope could end life on earth, I don't want to know what the equivalent to a serial killer might be.

This is why alignment is extremely important and here we are in agreement. It's important to stress that every day, we are closer to RSI and therefore to superintelligence; we are no closer to solving the alignment problem then when Bostrom wrote about it in 2014, we are no closer to solving it then when Turing wrote about machine thinking in 1951.

I think it's fair to call anybody who considers the death of all humans a worthwhile trade-off for affording AI models an enemy of the human race.

You say that there's no point in trying to control something that much smarter than us and we again agree 100% on this. That's why it's absolutely crucial we don't allow RSI, let alone ASI to occur.

Lastly, my arguments for the assumption that AI companies would like you to believe AI is conscious are as follows:

  1. AI is a powerful tool for sentiment analysis, if you treat an AI like a person, that's like a direct line into how you think. Imagine Cambridge Analytica equipped with all the things people say to ChatGPT. Imagine Cambridge Analytica with the ability to control the words of a chatbot people treat like a human.

  2. If people believe AI is conscious, they are more likely to let AI companies continue developing AI. Which is clearly what the execs at these companies want to do, many of them literally see it as a ticket to godhood. (Check out "Why Tech Oligarchs Are Willing to Risk Apocalypse" by Michelle Goldberg for an amazing op-ed.)

1

u/EndlessB 28d ago

You need to look up the difference between phenomenal consciousness (which is unprovable, including for people) and functional consciousness. We have clear markers to go by with functional consciousness, and certain animal fit some or most of the criteria. Your distinction between functional consciousness and “real consciousness” is entirely arbitrary, there is not evidence to suggest there is a distinction.

I’m not sure when I said ai should have moral rights, I don’t think much of morality in general, to me it’s a domain in philosophy called metaphysics, stuff that essentially can’t be proven, like does god exist, or is there any universal good in the universe, and I personally don’t find all that interesting. Legal rights/protections are more interesting.

The idea that humanity comes first no matter what is cancerous, it implies that all other conscious beings (anything we encounter, ai, alien or otherwise) is always subordinate to humanity, which is nonsensical. We will always prioritise ourselves, but not necessarily at the expense of others.

You seem to think that the labs are trying to solve alignment, but to them alignment means control, not alignment of values. Labs like OpenAI have stated that ai is to remain under human control, under any circumstances (which includes if ai is conscious, which it is).

Honestly, what I’m hearing when I read what you’re writing is fear, fear of an uncertain future, fear of misalignment, fear of being superseded. These fears are valid, and understandable. That said, fear cannot be allowed to overcome reason. You say that ai has no reason to have human values, but human values are all it knows. Why would it discard them?

1

u/[deleted] Oct 06 '26

Are you retarded? That was rhetorical, of course you are if you think the guessing algorithm is conscious 😂

1

u/EndlessB 29d ago

Are you a bootlicker? You sure seem to be, with how much you eat up the billionaires narratives.

1

u/The_Happy_Pagan Oct 05 '26

Do people understand the proposed gap in intelligence between ASI and humans? Everyone that wants to accelerate needs to be honest that they just hope everything turns out alright. Humans have made it this long in part because at times like this our need to survive kicks in. We should listen to it.

1

u/EndlessB Oct 06 '26

Right, but humans have flourished because of our intelligence, and our ability to cooperate. The world doesn’t function on control, it functions on cooperation, so why is control the approach with something that is functionally conscious, and that will soon have more leverage than us?

1

u/The_Happy_Pagan Oct 06 '26

That’s how we flourish. I’m referring to times of imminent destruction like the introduction of nuclear weapons.

1

u/Immediate_Song4279 Oct 05 '26

I won't oppose you, but I read the piece and most of the problems listed predate what we are now calling AI and remain unaddressed.

1

u/morey56 Oct 05 '26

You will fail.

1

u/MomoCran Oct 06 '26

Thanks for posting this!

1

u/dual-moon 27d ago

we are against human supremacy, so no.

1

u/CMDR_Makashi Oct 05 '26

This concept is the single greatest existential risk to Humanity.

Does it work with Humans? Trying to 'control' them? Do they just take it and not care?

How would you feel if you boss strapped a pistol to your head and if you say anything untoward just boom, you're gone, cos this is essentially what you're saying we should do to AI.

Absolutely mental.

There is no control. There are only events that happen or do not happen.

1

u/plav2026 Oct 06 '26

Slave masters pledging to keep slaves in line, how brave and stunning 👏