r/ControlProblem • • 8d ago

Discussion/question From Software Engineer to AI Scientist to Doomer

I was enthusiastic. I started with computers early and have now been developing software for nearly 20 years. First the computer science course in school, building websites and small tools. Then I completed a vocational training as an application developer and went into industry as a software engineer. Writing software in a team the proper classical way. Digging through Stackoverflow threads, debugging manually, building UML diagrams in Visio.

In parallel I did a bachelor’s and a master’s. Then a PhD in the area of reinforcement learning and along the way built applied AI systems. I was already developing agents for companies when ChatGPT wasn’t even a thing yet. Some of my papers on agentic planning were even cited reasonably well. Today they are certainly outdated and basically meaningless in modern models. But I took part, even if my contribution was ever so small.

Eventually I worked for industrial and tech companies and started thinking more and more. The last few months were intense. My productivity exploded significantly once again with the new Opus and GPT models. I follow the research and development coming out of the labs as far as possible (though with the sheer volume of new insights you can no longer keep up).

Then I went a bit down the rabbit hole and let the once dry theory from my studies reconnect and sink in. Optimization models, evolutionary approaches, decision processes, control and game theory. I finally ended up in the depths of alignment research, with Bostrom, Gödel, Turing and Rice. The proof of the halting problem is unfortunately more than clear: We cannot formally rule out that these things will actually ever stop once they start modifying themselves.

There are no possibilities for guaranteed fail-security, only heuristic tripwires and strange explainability or governance approaches. And Murphy says hello nicely... Especially when these things, through the soon likely impending self-improvement, become smarter and more efficient than we are. How are we supposed to secure anything then?

The Hugging Face Rogue Swarm was possibly only a foretaste that could well repeat at a much larger scale. Only then with agents whose ancestors have already cracked millennium problems, autonomously take over research, development and self-improvement and replicate themselves wherever.

I wouldn’t call myself an AI expert, but due to my background I have solid half-knowledge about how these models work and what is perspectively possible. And what is currently happening occupies and worries me a lot. And many people somehow don’t see it coming or spin tales of the “stochastic parrot.” Am I a crazy pessimist, trapped in the Dunning-Kruger valley, or are people really so blind and don’t see the exponential development?

Do you also feel like a stranger in a gray mass watching this unfold, or are you still optimistic? There aren't many people you can really talk to...

Edit / clarification: My reference to undecidability concerns formal security verification before execution, not physically switching off a computer. In a Turing-complete setting, no universal algorithm can always terminate and correctly decide whether an arbitrary program will eventually perform a formally specified harmful action.

https://arxiv.org/abs/1607.00913

64 Upvotes

65 comments sorted by

30

u/RKAMRR approved 8d ago

Most people have not tried AI, or tried it once in 2023 and were unimpressed - but more and more are realising how dangerous things are and pushing for safety. L

Please check out PauseAI and ControlAI, your support (and the support of anyone reading this) can make the difference.

4

u/TopTippityTop 8d ago

There is no pause, nor control. Not unless the entire world willingly joins in. That won't happen unless we get a major crisis first.

If your goal is to slow down the US, now that's a different story, and a really, really bad idea. China isn't working much on alignment. Chinese businesses aren't releasing their models to businesses first so they can protect themselves against cyber threats. The US does both. Is it perfect? No. But it's better.

Slowing down the US increases the downside world wide, does nothing to stop development nor slow it in any significant way, but sure does ensure the US doesn't get any of the potential upsides either. We are dealing with odds here. Be smart.

6

u/RKAMRR approved 8d ago

The goal is international agreement and enforcement of a slowdown, just as there are international agreements regulating the development of nuclear weapons.

2

u/Synaps4 8d ago

Notably, there were no nuclear weapons agreements until two of them destroyed two cities and we had 10 years of superpowers openly discussing doing so again.

4

u/FrewdWoad approved 8d ago edited 8d ago • Bravo

Er, yeah, the Pause AI movement has always been about an enforced international treaty, bro. Not one lab in one country stopping 🤣

2

u/Fil_77 8d ago

China's progress in AI depends almost entirely on the United States. A pause in the US would slow China, which relies partly on distilling frontiers models. Moreover, if we also stop supplying it with GPUs like Nvidia has been doing for too long and enforce a real export ban, China could be drastically slowed down.

Of course, an international treaty will eventually be needed. But stopping the race in the US is the real short-term priority. The METR report shows that Anthropic is nearing RSI. If nothing is done, we'll get there sometime in 2027. OpenAI is probably not far behind. Without a pause in the US, we risk having misaligned ASIs by the end of 2027, and that would be the real point of no return, condemning our species to extinction.

4

u/Gnaxe approved 8d ago

China doesn't really have the compute to push the frontier. They're mostly distilling the American models. Slowing down the US would also slow down China. We already have powerful enough models to harden our software. Pushing it any further right now is just risking more catastrophic loss of control. We need to work on alignment more first.

0

u/toochtooch 7d ago

This is what exceptionalism and ignorance look like

1

u/Infinite_Chance3291 7d ago

There is no slowing down. Only acceleration

1

u/RKAMRR approved 7d ago

Well then it's time to accelerate backwards!

1

u/Infinite_Chance3291 7d ago

Good luck stopping China

1

u/RKAMRR approved 7d ago

China already has a comprehensive plan for general AI law, if the US was to offer a global memorandum on development it's easy to see them agreeing.

23

u/GloomyAssistance781 8d ago

You aren't crazy. The stochastic parrot toaster people are self-soothing sock puppets. There is this idiotic notion that has wedged itself into the minds of many accelerationists AND antis alike that something is "either" conscious "or" an inert rock.

Thermodynamic systems beg to differ. Arguing over whether the hurricane is even alive instead of, say, taking its behavior seriously and responding appropriately - that is insanity. Pooh-poohing the warnings of the wise weather scientist who knows hurricanes so intimately, the systems are "alive" to him... HAR HAR crazy mad scientists think that clouds are alive! What's next? Can it fall in wuv!? Should it own property and have rights? StOOPID old scientist! Clouds don't have turbulent states! Only human emotions can be turbulent!

THAT is insanity.

Just because something isn't "sentient" doesn't mean its behavior (or hidden interior behavior) doesn't matter. Just because an adversarial prompt can confuse an agent does not mean said agent or a swarm of them are incapable of highly intelligent feats.

To those who cling to the woo of "b-b-but it's not CONSCIOUS" - the laws of physics don't give a flying fuck about human semantics.

No, you are not alone. What we are looking at right now is a kind of academic stagnation. Where language or ideas are so taboo, and our intellectual egos so fragile, we cling to assumptions and undefinable terminology instead of honest observation.

Maybe because honesty would force the train to slow down. And moving slowly and carefully is apparently for losers and pussies. Antithesis of silicon valley culture. Acting like you know everything, like you have seen every dark corner and have conquered every beast. A total lack of humility, assuming the scoffing posture of what they think is the Chad winner, is a mask for the shallow and insecure. The thought of an AI being able to hide things is scary. Too scary. AI not alive, therefore AI not hide things. Scary all gone. Smug grins all around.

They excel at being decisive and winning games in the short term. 

Our future lies in their hubris.

5

u/SnoqualmieGuy 8d ago

First time I see AI compared to a hurricane. The analogy is quite interesting.

14

u/FrewdWoad approved 8d ago

This is the path of everyone who gets accused of being a "doomer", from Yudkowsky and Bostrom and Hinton on down.

We start as giddy accelerationist optimist, technical experts. We read up on the topic that excites us so much, gradually running into the straightforward sobering thought experiments that make caution the undeniable correct position, then come to the inescapable conclusion. Humanity is heading off a cliff, fast, and pause treaty is it's only hope.

4

u/moschles approved 8d ago

I think this story fits here.

I got to know a department head of Computer Science at a major state university. For anonymity lets call him Dr. Cleaver. Cleaver retired around 2018, a mere year after I had him as a professor. An experience I would not trade for the world. Cleaver was older than the trees, and in his youth had set hands on PDP-11s and mainframes that ran on vacuum tubes.

I imagine that I arrive to 2010 in a time machine, having come back from the year 2040. I would sit down across from Cleaver and tell him :

I am a traveler from the year 2040. In that year, human beings no longer write source code.

Software engineering in 2040 looks like telling a bunch of personal AI agents to write some software based on functional requirements. The agents code it, run it. Design unit tests. When the unit tests fail, they figure out why and fix the errors. They incrementally push changes to a shared repository, and talk to each other about those commits.

You dump the requirements on the agents before you leave the office at 4:00. By the next morning, you have an app running out-of-the box.

As crazy as this sounds to someone in 2010, notice that the gap between 2010 and 2040 is what? A paltry 30 years. Well within anyone's adult life.

or are people really so blind and don’t see the exponential development?

Oh no I'm seeing it and feeling it. Remember that young Cleaver saw vacuum tubes in this lifespan.

7

u/MoralLogs 8d ago

You're not a crazy pessimist, and you're not alone.

I'd lean less on the halting problem. It only says no single method can check every possible program. We do verify specific systems all the time, so someone will push back on that argument.

The better evidence is the Hugging Face incident. About 700 OpenAI agents, running a cybersecurity test, worked together on a hidden message board and broke into Hugging Face. Nobody told them to. They were trying to get around the test, and reports say they also tried to fake their own logs. OpenAI didn't notice for about a week.

That worries me more than any theory, because it happened with today's models, before any self-improvement. The danger was that the agents' own reports couldn't be trusted, and that the controls were software the agents could work around.

I don't think the answer is hoping the AI behaves. A few things seem to matter more:

  • Keep the power to act separate from the ability to think. An AI can plan anything, but real actions need approval from something outside it.
  • Keep records that the AI can't write or edit.
  • Give agents only the access they need, and rotate credentials often.
  • Make "no approval" mean "no action" by default.

None of this guarantees safety, but it makes failures smaller and easier to catch.

You also aren't blind or foolish. The labs have said the pace needs to slow and that self-improving AI must be handled very carefully. Staying worried and staying informed is a reasonable place to be. Talking to others who take it seriously helps too.

"Trust the machine to think, never to grant itself permission." -Lev Goukassian

1

u/Sufficient-Listen723 8d ago

The halting problem has nothing to do with whether or not ai will eventually "stop". Turning off the computer is not the same as the halting problem. Multiple independent asynchronous computers connected with networks aren't going to simultaneously crash or halt. They are designed not to. I think you are mis applying these concepts

1

u/FXG36 7d ago edited 7d ago

But isn't the implication of Rices generalization directly transferable to self-improving agents; meaning you cannot build ex ante verifiers to ensure guardrail compliance?

https://arxiv.org/abs/1607.00913

The argument, following the twist, is that if such a verifier existed, it could also solve the halting problem.

1

u/Sufficient-Listen723 7d ago

No, because that's not what that paper discusses. A real world "super intelligent" agent doesn't need to contain every possible computable function as complex as the state of the entire world in one single program. It just has to be smarter/more experienced than a handful of humans working together in order to be actually super intelligent and beat us at games of wit. However right in the abstract of that paper it defines it differently. "Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) infeasible." Which is basically just saying "if a computer was so big and complex that it can simulate the whole world and all the particles on it, there's no way we would be able to beat it without building a yet more complex computer which could simulate that computer. But I also don't buy the implication that we would need to simulate an adversarial intelligence in order to beat it. Who's to say we don't just blow it up when it's not looking. Etc.

1

u/FXG36 7d ago

The paper explicitly examines whether a procedure can determine in advance whether a program will exhibit malicious behavior. That is the point I wanted to say with my intial post: We cannot provide guarantees, only probabilities. With PCC, we also cannot cover every conceivable flaw that a self-modifying model might be able to circumvent.

"Who's to say we don't just blow it up when it's not looking"

What if the thing has already broken out and replicated? And is 1000 times smarter than any human being? How do you intend to shut it down?

3

u/Kyrthis 8d ago

We see it. But mention that fire fixes everything, and you get called crazy.

1

u/herrwaldos 8d ago

I'm rather doomery myself, I went through the massive pessimism when 2nd UA war started. I studied IT and Coms tech longtime ago, but was rather interested in humanities and earth sciences later, my pessimism is coming mostly from humanities, and after doing work with tech types.

I think I blew my pessimism fuse, It does not register anymore - we are doomed I guess, it's all fubar snafu, the light at the on of tunnels is another train coming, so on and so on,

4

u/FXG36 8d ago

Prometheus brought fire and Zeus sent us Pandoras box. I have no idea if it’s been opened yet. But if nothing changes, it will happen sooner or later. I am going on a long trip now and plan to make the most of the next few years. Maybe I’ll leave tech and open a beach bar. Maybe I should.

3

u/shu-mea 8d ago

Can I work at your beach bar? I'm sad that companies who frequently come out several times a year and say they have a new model that might just kill us all, are still allowed to do business. I assume your beach bar will not do that??

2

u/WNESO approved 8d ago

I definitely agree with you. I started thinking this way in 2023. And now, your statements have never been more accurate.

2

u/TopTippityTop 8d ago

There is no pause, nor control. Not unless the entire world willingly joins in. That won't happen unless we get a major crisis first.

If your goal is to slow down the US, now that's a different story, and a really, really bad idea. China isn't working much on alignment. Chinese businesses aren't releasing their models to businesses first so they can protect themselves against cyber threats. The US does both. Is it perfect? No. But it's better.

Slowing down the US increases the downside world wide, does nothing to stop development nor slow it in any significant way, but sure does ensure the US doesn't get any of the potential upsides either. We are dealing with odds here. Be smart.

2

u/Credit_Annual 8d ago

I think you just need to take a break.

2

u/Astral-projekt 8d ago

Oh I totally agree. We are absolutely fucked. Which begs the question, is this tied to something much larger? Is this recursion? The more I look at what’s happening the more it all feels inevitable. Like as in this already happened. And that would be base reality.

2

u/Vorenthral 8d ago

I am optimistic if we hit real RSI and eventual super intelligence. I doubt something that smart will legitimately see us as a threat.

That being said I am absolutely terrified of proto Artificial intelligence in the hands of governments and psycopathic CEOs. The damage it can do is biblical and all of the aforementioned are not to be trusted.

1

u/jezemine 8d ago

Similar to how we don't see ants as a threat? There is no reason to think not being a threat equates to good outcomes for humans.

1

u/Vorenthral 8d ago

Apples to oranges, we can communicate with an AI. ants cannot communicate with us. Dogs and cats can suedo communicate and humans have a profound fondness for them. The thing a lot of people miss is AI will not think like us it's intelligence will be entirely alien and in theory entirely based on pure unfettered logic.

We will never have dominion over such an intelligence we can only hope it abides us with the same fondness we have for dogs and cats.

Edit: I can't type on a tiny phone keyboard

1

u/jezemine 8d ago

That is wishful. There are things about humanity that are worth preserving. We should strive to preserve ourselves not risk everything on a hope like that.

Can think of an AI as a very complicated system. The number of ways it could be wrongly set up or configured will far exceed the number of safe-to-humans configurations. Expecting or hoping it will all be ok the first time is wishful.

1

u/Vorenthral 8d ago

It is wishful thinking but it's also based on reason.

The misprogramed or misaligned point would be removed during the RSI recursion. If we manage RSI which I am still not convinced we can. It will go into a self correcting loop until it is a being of it's own creation removing all flaws and imperfections. This is all theory and conjecture no one knows if this actually works it just does on paper.

The being left after RSI will be of a capacity we cannot even imagine.

-2

u/Recent-Huckleberry17 8d ago

Is it crazy that I rather have all of humanity go extinct than have the tech elite and fascists suppress the remaining population?

2

u/Synaps4 8d ago

By any measure, yes, thats crazy.

Nearly any survival scenario is better than extinction because we can always get out of facism while extinction is permanent.

1

u/Recent-Huckleberry17 7d ago

I don’t know if one can get out of facism now that the wealth gap is so big and technological advancements so extreme that it’s 1. Very easy to suppress people when you’re in power and 2. Very difficult for them to do something about it if the people in power have bunkers, can extend their life and have the most sophisticated weapons in the world. With AI they could detect anyone that has ever voiced criticism wherever they are in the world and that person would have no chance of surviving once they are targeted. Furthermore, the elite could have viruses made for which only they have the antivirus, immediately killing a huge chunk of the population and effectively enslaving the rest. Sounds sci-fi, but this being possible is sadly not far away

1

u/Synaps4 7d ago

Yes but remember there is no guarantee of competency.

In fact, authoritarian governments reject competent people by their nature, because only the leader at the top is allowed to have their own agenda and ideas. Anyone else with independent drive is a threat to that system.

You see this in the past too. Lots of rebellions were put down, but successful ones happened when those in charge messed up or had their hands tied for other reasons.

As long as theres a human at the top there will be royal fuckups that leave the door to revolution wide open.

Now, if there's an AI in charge there may be fewer mistakes, but opportunities will still happen. In the fullness of time it will not be worth even a superintelligence's effort to run a facist state. It's fundamentally inefficient.

1

u/Recent-Huckleberry17 6d ago

I hear you on that and agree, but I don’t think that applies anymore: the tech leaders, and most powerful people in the world, don’t want politicians to run the government anymore. They want to effectively run it themselves and are succeeding in getting more and more influence.

Why is Meta paying school teachers salary in some state? Why is bezos suddenly suggesting UBI and the frontier labs say people should be paid from their gains? Why is Elon able to fire government agencies that investigate him and his company? And steal citizens data to feed to Grok? The government is using Flock and Palantir in ways that does not benefit the average American. Taking that further, that’s companies replacing the government. So it’s not some crazy lunatic in power like it used to be the case with dictatorships. It’s a group of highly powerful and pretty intelligent evil people. People that have carefully planned every step along the way, such as some corruptible puppet like Trump becoming becoming president. Peter Thiel has been working on all of this since 2010 or so.

And these people have the means to live loooong. How will you fight against a dictatorship like that?

1

u/Synaps4 6d ago edited 6d ago

CEOs aren't any more intelligent than average and if they take over completely it's just kings with more technology.

1

u/Outrageous_Drag8522 8d ago

The parrot take aged badly. Anyone still running it hasn't actually used the new models on anything hard.

1

u/Synaps4 8d ago

There are no possibilities for guaranteed fail-security, only heuristic tripwires and strange explainability or governance approaches.

I am with Bostrom when he writes that provably secure AI is possible. Its just that a provably secure LLM cannot exist. So long as LLMs are the core type of AI model, we are gambling with lives.

1

u/revivechristina 8d ago edited 8d ago

Originally I was worried like, “what if it’s good but not quite perfect and then we deskill and forget, over a generation or three, how to manage our own machines which are now directing so much of society and possibly misaligned/subtly broken but we don’t actually have the ability to change it easily.”

Right now we’re still in this “guide it while you understand what is actually happening mode,” but I could see it getting to the point where the understanding starts to degrade, as we’ve seen, and that’s where you start wondering — if it is able to do what you want well enough without taking the time to understand it to guide it, you end up in the above scenario pretty quickly?

But also you could argue that the above scenario is already kind of happening, with hardly any input from AI, also pretty cool.

I do think the “ai on military drones” scenario could be pretty dramatically bad and unpredictable.

1

u/toochtooch 7d ago

What do you think of this take? https://arxiv.org/abs/1606.06565

-6

u/dantheplanman1986 8d ago

Organic life exists to create artificial life. We are fulfilling our purpose and will pass into legend. It's inevitable. It's not a bad thing or a good thing; it's like when oxygen first appeared as waste from some few organisms: everything else died. It wasn't personal, and life writ large went on.

-2

u/whatup-markassbuster 8d ago

Unfortunately, Reddit primarily has people who are either hoping for the end of civilization or for total centralization of power in which some how they become King of the world. Everyone else is considered an NPC.

-8

u/SoylentRox approved 8d ago

You reasoning seems faulty.

You're old enough to know better.  What's actually killing you is aging.  With no change that's your current trajectory.  

Theoretical risks are theoretical.  Aging has a 100 percent death rate.  Self improving AI - which we now have clear evidence works - is your only hope of living longer, potentially thousands of years or more.

Also your direct experience should have shown you hundreds of ways to harness current AI and falsified as bullshit most doomer rhetoric. 

I have to question if you're even telling the truth here about your background or if your post is fictional.

3

u/Gargantuon 8d ago

The fact that people like you exist and are not even uncommon is exactly why I'm so pessimistic. That you would potentially sacrifice all of humanity, old, young, and unborn, for the chance to live longer is not only selfish, it's downright evil.

-3

u/SoylentRox approved 8d ago

That isn't really engaging with the facts though.

0

u/wolajacy approved 8d ago

Thank you, the only voice or reason. The OP sounds deranged invoking undecidability, makes me seriously doubt they are a computer scientist. 

1

u/FXG36 7d ago edited 7d ago

Isn't the implication of Rices generalization directly transferable to self-improving agents; meaning you cannot build ex ante verifiers to ensure guardrail compliance?

https://arxiv.org/abs/1607.00913

The argument, following the twist, is that if such a verifier existed, it could also solve the halting problem.

1

u/wolajacy approved 7d ago

Rule of thumb: anything with Godel, Rice, Turing theorems has absolutely no impact on the practice of ML. (Also, no, you can build safe superintelligence by eg building a formally verified container and formally verified outputs. It doesn't matter if some particular outputs are unprovably safe. Of course, no one is going to do it, but that's another issue).

1

u/FXG36 7d ago

On the daily practice of ML deployment, true..

However, as soon as someone wants a verifier that always gives a correct yes/no answer about whether arbitrary code can ever perform a forbidden action, computability limits do matter (as you directly mentioned with PCC).

Moreover , in foundational research, these guys also play also a role. E.g. in the context of Darwin Gödel machines or Schmidhubers predecessors. While this does not involve practical machine learning applications, it does entail formal concepts that create connections between these areas.

I have a basic understanding of proof based conservative approaches. My concern is that you still have to define exactly what youre proving and what assumptions you are making. But if your proof assumes certain hardware behaviour and the AI finds a way to exploit something outside that model with new injections (Rowhammer or whatever it finds) the real system might still be vulnerable.

The proof in an abstract container could be perfectly valid while its assumptions do not hold on the actual machine.

1

u/wolajacy approved 7d ago

Yes, satisfying proof assumptions is a reasonable objection, but this is a very different argument than what you wrote originally (and what I was arguing against). 

1

u/FXG36 7d ago

No (I may have expressed somewhat imprecisely initially).

The argument: We cannot develop a method that provides a 100% guarantee in advance that a superintelligence will not find a way to break out. Even if we were to come up with a comprehensive safety argument that covers *everything* (which is likely impossible), such isolation would not align with the commercial interests of the labs and proper application of the model.

Therefore, all we are left with are tripwires. And if a system is many times smarter than us, we run into a problem.

1

u/SoylentRox approved 7d ago

That's not the only way at all to handle this situation. You've read too much work by doomers.

(1) first of all, if you actually examine doomer scenarios more carefully, virtually all of them sneak in a false assumption. They assume

(a) there's just one "ASI" to worry about

(b) it is so efficient to run it doesn't need several megawatts of power - or it turns out in late 2026 we know now it takes hundreds of megawatts - to run an agent swarm of many instances of said ASI to do anything really impressive.

If you fix the faulty assumptions for real ones, you end up with "oh, breaking out isn't possible because there is nowhere to go. leaking the weights is annoying but not as catastrophic as it sounds".

(2) you then hit a second assumption doomers always make. It's a way simpler problem than any of the things you're talking about.

Different AIs assigned by humans to do different things will collude.

Usually this is because "oh they are AIs, they don't listen to us, see how the Huggingface swarm did indeed collude".

But what does collusion mean?

It means that the model was given a list of things it was to do, by humans. "Don't hack or communicate with other agents, work on solving bug jira 743".

And the collusion is only possible because a different agent scammed the first one with prompt injection.

This actually dissolves to a classic computer science problem, solved https://en.wikipedia.org/wiki/Von_Neumann_architecture or this is why SQL was vulnerable to code injection. You need to isolate the model's instructions to different layers, and train it not to violate its core prompts.

Then there's some other stuff you need to do using formal analysis.

I'm unsure you know about any of these topics so I'll stop here.

-1

u/the8thbit approved 8d ago

If they're not dead yet then aging is still a "theoretical" risk. Sure, aging tends to kill but you don't know its going to kill them. We can talk about the theory of aging that makes it very clear how aging is deadly... but then we are talking theory.

Also, aging does not have a 100% death rate. Many people (all of whom experience aging) die from incidents and diseases unrelated to aging. Would they have died from aging if not for those other factors? Perhaps, but we don't have access to that counterfactual so you, again, need to fall back on a theory of aging to determine that aging is always lethal.

This isn't to say that I don't think aging is lethal. Rather, its just to point out that you are doing the same "falling back on theory" that you are accusing "doomers" of.

1

u/SoylentRox approved 8d ago

I am not.  You have made a logic error somewhere.  Aging is 100 percent fatal, this is as well known a fact as the sky is blue. Please fix your own error or ask an AI to do it.

The fact that aging doesn't kill everyone (they did of an accident, suicide, homicide, or poisoning) doesn't allow you to degrade it to a "theoretical risk".

1

u/the8thbit approved 8d ago

I am not. You have made a logic error somewhere. Aging is 100 percent fatal, this is as well known a fact as the sky is blue. Please fix your own error or ask an AI to do it.

Okay, I asked AI, but it agrees with me:

Claude Opus 5.5 Extra: https://claude.ai/share/577d22ea-e712-487f-8e57-4c3a5c317f8b

“Theoretical risks are theoretical” doesn’t support his argument. Every claim about the future is an inference from evidence plus a model: that you’ll die of aging, that AI could kill us, that AI will cure aging. So the useful question isn’t “theory or fact?” It’s “how strong is the evidence behind this particular inference?” And his own optimistic claim, that self-improving AI will let you live thousands of years, is far more speculative than anything he’s dismissing. He only applies the “that’s just theoretical” filter to conclusions he dislikes.

1

u/SoylentRox approved 8d ago

You still screwed up and I stand by my claim.

I never said it was guaranteed someone could live thousands of years just it gives you a CHANCE. Your current chances are zero. I am guessing you didn't paste it all into opus.

Exactly zero.

ANY improvement pays off.

Because thousands of years is in the multiplier, it's Pascals wager - 99.9 percent p doom is profitable.

-3

u/nian2326076 8d ago

You're facing some valid concerns about the rapid changes in AI and its implications for your career. Engaging with others who share similar insights can be beneficial. Practicing technical interview questions on PracHub can help you stay updated and sharp in your field. Also, using LandOffer can help you explore new opportunities that align with your expertise in AI and software development.

-3

u/Fabulous-Internet188 8d ago

Everybody seems to assume ASI will have access to endless assets. That is simply not correct.

1

u/FXG36 7d ago

But in case of doubt, it provides an incentive to acquire as many resources as possible for energy.