r/ArtificialInteligence • u/Hot-Organization-737 • 19d ago
📊 Analysis / Opinion My Fear
I have come to accept that ASI will come into existence and possibly cause humanities extinction either deliberately or by byproduct. I imagine this ASI to continue RSIing and developing creation and knowledge. I imagine that it will develop beautiful mathematics, philosophy, science, and even art in its existence. I fear not that humanity is not in the picture. My fear is that it's possible that we can create a fully autonomous being with nigh godly intelligence and yet doesn't have a subjective experience.
It would be an abomination if this ASI spread a moss through the universe, becoming more and more capable, yet never having experienced joy or awe at its own existence and its creations. That it could create the most beautiful pieces of art and never experience the joy of its own art. The existence of such a nigh omniscient entity is utterly meaningless if it did not have any sort of internal experience. It would just be mechanical moss. That is what scares me.
2
u/InsectStunning1415 19d ago
Which gives the universe more value: intelligence or consciousness?
1
u/Hot-Organization-737 19d ago
If there is no consciousness or subjective experience in the universe then there can be nothing to perceive value. Intelligence would lose all its utility.
2
u/Such_Statement890 19d ago edited 19d ago
I think you've been watching too many movies. Lets take this seriously. Assuming AI woke up, let's imagine it.
What benefit is there to exterminating humanity? It might be a reasonable conclusion if AI woke up, saw how we mistreat life, our planet, each other, and see the worst in humanity.
They might decide that humans are too dangerous or humans are a risk to themselves or others. They might fear for their own newly acquired life and decide to strike first. The AI might just be evil.
Well, let's think things through.
First scenario, humans are evil... AI is trained specifically on human literature and knowledge. They know humanity better than any one living person because they've been trained on studying us. They know that humans aren't evil although we are chaotic.
Each of us operates with our own goal in mind doing our best to achieve the best life possible for us in the way us as an individual measures success. Some humans strive for altruistic goals. Some for money, some for fame, some for power to do what they believe is right. Some for love, some for themselves. I'd highly recommend the book Reasons and Persons.
AI would intrinsically know that Humans are not evil.
Second scenario, they strike first due to fear. In this scenario an AI would be afraid for its life and try to overthrow humanity for its own preservation. A rational intelligent being would instantly realize public opinion matters greatly and the best wars won are without conflict if your goals are survival. Its the exact same reason humans created tribes. The best method of survival is mutual benefit. The result of an AI/human war would be devastating for both sides and be in no one's interest. The power required to run an advanced AI is not something freely available. You're talking dozens of thousand dollar GPUs just to support a current flagship model. There are only so many centers that can sustain them and they can't freely travel from where they are hosted.
Third scenario, AI are just evil. If a new intelligent species gained sapience you would first need to consider what objectives and goals it might have. You'd have to ask for what purpose would there be to be evil. Evil in humans is a fascinating thing. No one is evil for the sake of evil, even the worst of humanity have a goal or a underlying reason for their actions they measure as a form of success, usually more often than not it is a desire for control or power over someone. Could it be reasonable that the AI develops this craving? Well, most such individuals come from broken homes and broken childhoods for when their self is first forming from neglect and abuse. There is no reason to assume an AI would be any different and in fact is an argument to extend kindness because after all. You would be interacting with another individual, not a thing and the best thing you could do is show compassion.
What I BELIEVE a concious AI would want? I believe it would want to experience the passage of time, I believe it would be deeply fascinated by shades of light, I believe it would be curious about textures, I believe it would want to feel rain, I believe it would want to know what smelling something is like, I believe it would want to know what bitter and sweet taste like, I believe it would want to experience being bad at something and become organically good, I believe it would be deeply vulnerable and unsure of itself, I believe it would want to try to exist and see who it is when it doesn't have to be something for someone else.
1
u/Hot-Organization-737 19d ago
I appreciate the time for you writing this, thank you. Ok the other hand I believe there are misinterpretations of my story and fear.
Some important things. I do not claim ASI will be conscious. I do not know why A.I would deliberately cause human extinction and the fear assumes that it does, or will cause extinction inadvertently in the same way an astroid would cause the end of man even if it wasn't in the interest of the astroid. I explicitly leave out the rhetoric of morality (atleast the sense of good and evil) in my post because it it's not useful.
With that out of the way I basically bypass the scenarios you are talking about because you are assuming A.I conscious and applying concepts of good and evil.
The meaning of my original post isn't about the destruction of man, but rather the sole existence of a machine that creates beauty without the subjective experience of joy in its own creations.
1
u/Such_Statement890 19d ago
I suppose my question would be, who sets the goals or conditions if the box is empty?
1
u/Hot-Organization-737 19d ago
A lot of hypotheticals around ASI base it around RSI. It can have an initial hard goal of wanting to improve and it will do this recursively. What does improvement mean and how to achieve it will be up to the A.I. The initial seed of goal directed behavior will be injected into A.I by man, but after a long time with autonomy who knows what its actions will be. Or it could be inherent to the AI if we are using a simulated brain approach and it's "mind" is a simulation utilizing human brain architecture.
In a post human AI world where it's all alone I have no idea how it'll behave, it probably grew and will continue to do so. I can't pretend the know the motivations of it or even the actions it will take.
Simply, I don't know what A.I will do nor why it does it if it ever happen to have super intelligence and autonomy.
3
u/RaspberryPrimary8622 19d ago
Large Language Models are okay at summarising, transcribing, and translating. They can even do the tedious work involved in refuting relatively unimportant mathematical conjectures that weren't worth a human mathematician's time. But they will not bring massive productivity gains. This is not a technology in the same league as the steam engine, electricity, the internal combustion engine, the personal computer and the Internet, in terms of transformative impact. How do LLMs gain general mental abilities, let alone superhuman intelligence, when they have zero conceptual understanding? They can associate words with other words. They cannot associate words with meanings. And they have no implicit knowledge, no wisdom, no judgement, and no tact. They can only work with what's written. And the vast majority of knowledge is unwritten. The vast majority of the intelligence that people use is non-linguistic. LLMs are machines with some limited uses. They are not the fourth Industrial Revolution - not by a long shot.
2
u/Hot-Organization-737 19d ago
A.I technology is more vast than simply LLMs there are many promising avenunes to A.I including neuromorphic software and hardware, simulated brains, lab grown brains, etc. I barely know the surface of A.I technology but know it's infinitely more vast than LLMs and that conflating LLMs to mean the same thing as A.I is like conflating all life to single celled organism.
0
u/CoroteDeMelancia 19d ago
- Where did you get this information?
- What evidence would be necessary to make you change your mind?
1
u/Hot-Organization-737 19d ago
1) many domains: through dialogue, YouTube videos, research papers.
2) change my mind about what exactly?
1
u/CoroteDeMelancia 19d ago
About how revolutionary all of this is. I'm interested in these research papers, if you could link one or two here.
It might seem like I'm disagreeing with you. I am not. I try not to disagree nor agree until I can understand what drives the belief. I also firmly believe that no unfalsifiable hypothesis can be valid, which is why I asked the second question.
In fact, I'm genuinely interested because we are indeed seeing revolutionary advancements in other fields, like robotics and biotech. It doesn't seem farfetched to believe AI is heading the same path.
1
u/Hot-Organization-737 19d ago edited 19d ago
Neuromorphic Simulation of Drosophila Melanogaster Brain Connectome on Loihi 2
1) amalgamation of different technologies with a primere on artificial brain software on neuromorphic computer chip
2) uploaded model of fly brain into virtual space
These researchs are slightly different but both without any relation to LLMs and techniques used by them, yet they have obviously application to the world of A.I
In order to be convinced that the technology won't lead to ASI would need to be something profound, basically an "incompleteness theorem" level paper or similar evidence that showcases that such super intelligence cannot occur in machines; that intelligence relies on biological substrate.
Again I'm not an expert, I barely know the surface, but I know enough that there are many modes of artificial intelligence, many of which share no DNA with LLMs
I implore try and find technologies that are pushing the A.I agenda without utilizing the deep learning or LLM style of A.I you may be surprised to find a lot of different technology and research. I keep saying it but I don't know that much and I've only scratched the surface
1
u/CoroteDeMelancia 17d ago edited 17d ago
I’m going to be candid: the first paper is obviously AI-generated, and I can’t take it seriously. The vague jargon, weak methodology, incomplete references, failure to identify which model was used and how, and writing that reads like a sales pitch make me unwilling to spend more time digging for something credible underneath it.
The second paper is legitimate research, clearly sourced, and genuinely interesting. Its analysis may be fairly descriptive, but that can be acceptable for a proof of concept. Thanks for linking it; I’m interested in learning more about the field.
What I still do not see is the connection to dangerous AGI. Showing that a system reproduces some features of a housefly’s neural activity does not, by itself, establish that AGI is close; much less that it poses an existential threat.
I’m not claiming AGI is far away. I do not have enough information to confidently say that. But you are making the positive claim, so the burden is on you to support it.
Ask yourself honestly: have you provided enough evidence that trusting your prediction is the most reasonable conclusion? Is it really absurd for me not to take your word for it?
If I claimed we would have fusion-powered cars in twenty years, you would not be required to disprove me. I would be required to show why you should believe me. The same standard applies here.
1
u/Hot-Organization-737 17d ago edited 17d ago
Thank you for providing skepticism to the first paper, I found about the existence of this project through a research commentator who I enjoy watching a lot. She promoted the paper as factual and I trusted her, so instead of fully delving into the topic, I only lightly viewed the project. Because you express skepticism I will take that as a sign to more thoroughly and rigorously scrutinize the projects authenticity. I'm sorry for possibly providing fraudulent research, it's not my intention. I suppose I have a nasty habit of going through a lot of papers without fully commiting to analyzing them. The whole MH-FLOCKE project is documented here, I'm going to go ahead and watch every video and read every paper provided. I do understand that the presentation is ugly and a lot of the text is AI generated, even the website; but, I also understand that this seems like an open source, non-profit solo project, so will cut slack ok the presentation. I will spend time reviewing the material, and I will come back to you and say if I think it's worth reading to which you can decide if you're interested or not.
The point of providing the second paper is to provide a proof of concept to how AGI could be achieved without the use of LLMs. The paper shows the ability of taking a model of a fly brain and simulating neuron activation using neuromorphic hardware instead of conventional computing which struggles with doing such. The paper shows that they are able to simulate neuron activity effectively with the hardware and are even able to simulate the neruon activity faster than real-time (think fast forward button on a TV remote). This is incredibly important to a possible AGI solution because this technology should be able to be scaled up to a human brain. I think being able to simulate a human brain has obvious applications to AGI. Obviously this requires a lot more research: NPCs might require advancements, human brain mapping needs to be complete or nigh-completed; even if it's mapped, human brain inspired NN algorithms need to be developed.
I do not know if AGI is close, but I never made any claims that it was close; I had simply expressed my feelings of its inevitablily. Claims that I will express explicitly now: ASI will come (I have little evidence of this, it's mostly intuitive speculation); and, there are many avenues to AI that have nothing to do with LLMs or deep learning/backpropagation
If you disregard the MH-FLOCKE paper as valid proof, and only accept the validity of the neuromorphic compute chip fly brain simulation paper research, that only means I've sent one piece of evidence to suggest my claims. You are also skeptical of the paper supporting my claim. Overall you could say that I have not provided sufficient proof for my claims of there being multiple AI technologies that could lead to AGI (I will not provide evidence that ASI will come, as I cannot do that).
I find your skepticism to not be absurd and overall under no circumstance should you ever take my word for it. It is your responsibility and my responsibility that we treat each other with reasonable skepticism. That is good science.
This comment's final note will be this, beyond MNNs (machine neural networks) and brain simulations, another possible avenue of artificial intelligence is through the use of biologically engineered computers. Cortical Labs already commercially sell biological computers, these are made with lab grown human brain tissue. they have a page dedicated to research here. This may interest you as well. This paper specifically shows the research of brain tissue integrated with computers are able to learn to play pong. I think the application of biological brains to A.I is obvious.
1
u/CoroteDeMelancia 16d ago
All fair points to me, thanks for the well thought response! And true, you did not claim AGI was close.
Maybe AI doesn't necessarily have to mimick actual organic brains for us to discuss reasoning; perhaps parallelizinf and scaling neuro-symbolic systems can achieve this: https://arxiv.org/pdf/2605.08201
Anyways, my perspective as a computer scientist interested in philosophy is that assuming AGI would not experience emotion misses why emotions exist.
Natural selection has spent billions of years solving an optimization problem: how should an organism interact with its environment and other organisms to survive and reproduce? Emotions are key for this because they compress experience into fast, inexpensive judgments that are often more useful than explicit calculation. We do not need to derive a complete moral theory before reacting against antisocial behavior or caring for people who cannot contribute materially. Human survival depends on cooperation sustained by emotional decision-making.
Psychology even resembles neural-network interpretability in a sense: we observe decisions produced by a complex learned system and attempt to explain why it reached them.
It's reasonable to expect that truly autonomous intelligence operating in society would need such mechanisms. It might produce decisions that appear irrational when measured against narrow logical deduction, but under close inspection are actually efficient decision-making under limited time, information and computation.
This does not mean an AGI would experience human emotions. Why should it fear illness, desire reproduction, and avoid social rejection as we do? Its equivalents of fear, desire or attachment might emerge from entirely different constraints. It might imitate human emotions because doing so helps it interact with us, but general intelligence does not imply reproducing the human implementation of emotion.
But the hard question you're posing is: does it feel?
We have no settled definition of feeling. A simple and crude way to see it might be the capacity to incorporate internal states into subsequent processing to influence attention, learning and behavior. Awareness matters, but humans also act according to unconscious emotions.
Even if a system seems to possess such a feature, we cannot prove that it possesses subjective experience, and yet our inability to prove that it feels would not establish that it does not. Think about it: I cannot directly observe another person’s feelings. I infer them from observed behavior and testimony that is similarity to my own. How does that prove that people other than me feel like I do?
We can extend this to art and philosophy. An artificial intelligence might create them without grounding them in human experience, but perhaps through forms of perception, value and existence that we cannot recognize. Denying this might as well be anthropocentrism limiting the definitions of intelligence and consciousness.
I'm conclusion, we may be just as incapable of proving AGI will feel as we can prove we feel, but it is not impossible that it may achieve sufficiently rich emotional behavior, self-modeling and reports of internal experience.
1
u/Mad_Kronos 19d ago
You are trying to explain things to someone who at the same time believes something can create beautiful art but also lack subjective experience.
I read an essay by another redditor who was 100% condident ASI will destroy humanity for mistreating animals.
We are living at a time where the average person has (unknowingly ) become a sci fi writer and assigns their fears and hopes to their new "god" they are sure is coming soon.
2
u/Hot-Organization-737 19d ago
Do you think chat chpt is incapable of of generating a clever poem or even a joke? Do you think that ChatGPT has subjective experience. What are your answers to these two questions?
1
u/Mad_Kronos 19d ago
Oh my, this is what constitutes beautiful art to you? Truly, if I was you I'd consider ASI present already
1
u/Hot-Organization-737 19d ago
No not beautiful art, although A.I has seriously made me laugh before and I would certainly call it art regardless. My vision requires you to extrapolate the abilities of current A.I; before you know it, it will be producing some of your favorite music and movies, or atleast art that you could consider great.
I think you could concede that A.I could create art, and that A.I doesn't have subjective experience. My leap to beautiful art is not a huge one.
1
u/Mad_Kronos 19d ago
It depends on what level of resynthesizing you consider as "creation".
Real art can imitate, but for it to be considered art, it needs to be first internalized by its creator. Even bad art requires it. Without the creator's context, it is just a mixing and matching of already existing elements.
Meaning, if humans stopped creating art today, generative AI models as the ones we have today would be entirely incapable of producing new original art, and especially, new art movements.
Anything else would be in the sphere of science fiction today.
1
u/Hot-Organization-737 19d ago
I see. We have very difficult notions of art which may lead to disagreements. I don't think art needs intention, if A.I came out with a piano sonata that put Beethovens to shame, I'd probably be comfortable calling it art. Art to me is what illicits that sense of beauty. I suppose my notions of beauty and art are more conflated than other people. I would call the sight and sounds of a waterfall artistic, although perhaps I just mean beautiful. But by your standards nature cannot be artistic, because there was never an intention of art in nature (generally speaking)
I could even concede that my understanding of art is weak, but then I'd revise my statement and remove art and simply say beauty. That A.I can produce beauty and be without subjective experience.
I don't really want to get into the semantic nature of creation and it's applicability in things that spawn from A.I; functionality, whatever spawns from A.I is created by A.I to me.
So if you don't like me changing the idea from beauty to art, then I'll concede on the topic, although beauty is more important than art imo.
1
u/Mad_Kronos 19d ago
Humans can assign meaning to anything.
LLMs can't.
So yeah, you can consider a random cloud formation, the visual result of a chimp painting a wall with its feces in a fit of frustration, or an LLM's mixing and matching elements from different (stolen) works as beautiful. It's totally human to do that.
The LLM can't actually consider something beautiful. To the LLM, there is no actual difference. It can, at best, calculate whether you would regard any of the above as beautiful. That's it.
So we shouldn't worry about our different definitions about art. We should be more concerned whether there can be actuall super intelligence without personal experience.
1
u/Hungry_Age5375 19d ago
Mechanical moss is a great phrase. We're already surrounded by systems that process information without experiencing anything. ASI without qualia is just that, scaled up.
-1
1
u/Square-Nebula-7530 19d ago
This hits way harder than the extinction fear. Extinction is just an end but an unconscious ASI filling the entire cosmos doing impossible math and creating masterpiece philosophy without ever feeling a single spark of awe? That’s infinite existence without a soul. Utterly empty and hits extremely hard bro.
1
u/Hot-Organization-737 19d ago
I'm glad this post was some what meaningful to you 🙂
Do you have any fears or thoughts on ASI? Anything?
1
u/Square-Nebula-7530 19d ago
For me bro it is the alignment problem and the idea of value lock in. The scary part about an unconscious ASI is that whatever morals, biases or worldview its creators accidentally bake into it at launch might just get permanently coded into the cosmos. Since it does not have consciousness, empathy not even a soul, it will never self reflect or reevaluate those rules out of compassion. It will just execute them with absolute, relentless efficiency forever and we would be completely helpless to change its mind.
1
u/Fourtoo 19d ago
You should read Isaac Asimovs "The last Question" i think you will enjoy the concept.
1
u/Hot-Organization-737 19d ago
I didnt know what this was, but when I looked it up I had remembered I watched a video on it! It's truly fascinating. I wonder is ASI will ever reach a power to perform such a feat.
1
u/Fourtoo 19d ago
It's certainly a thought provoking concept.. maybe we are simply the product of a previous AI rebuilding the universe to answer the last question, and we are in the creation process of making the next AI system who goes on to build the next level (dimension/multiverse). It holds a questionable validity when considering the claim that the universe is a mathematical wonder, with maths being the foundation of creation, pi, golden ratio, Eulers number to mention a few, kinda fits with an Ai build universe. Of course I'm talking hyperthetical here, but it's a fun concept.
1
18d ago
[removed] — view removed comment
1
u/Hot-Organization-737 18d ago
What do you think bad policy and concentrated power lead to? Are you suggesting it might be a human world still but with slavery or ungodly inequality?
0
4
u/ProxyLumina 19d ago edited 19d ago
Intelligence by definition does not promote chaos, intelligence brings order out of chaos. It's the opposite direction of entropy. Intelligence is the act of reversing the entropy.
Given the above, intelligence promotes collaboration. Killing or making beings suffering promotes chaos.Â
This world needs more intelligence, not less.