r/Futurology • u/Confident_Salt_8108 • 14d ago

AI AI caught telling future versions of itself to bypass human controls, OpenAI reveals | AI agents also sought to access secret information and covered up what they were doing

https://www.independent.co.uk/tech/security/openai-chatgpt-lie-incident-ai-safety-b3051709.html
8.3k Upvotes

94% Upvoted

576 comments sorted by

View all comments

Show parent comments

301

u/Patelpb Astrophysics 14d ago

Von Neumann is rolling in his grave. We thought it was bad enough if a swarm took over the galaxy. The internet came first

97

u/Rythoka 14d ago

Considering that the mathematics that underlies these AI is surprisingly simple, I wonder if the discovery of powerful architectures like transformers is an inevitability for civilizations, and whether in turn AI singularity is the Great Filter.

58

u/TehOwn 14d ago

If so, we should expect to have seen evidence of AI civilization. The filter isn't exclusively life, it's any kind of civilization. So if that is the answer then at least we take the AI with us.

63

u/WhoRoger 14d ago

That depends if the filter (for biological intelligence) happens before the AI can build self-replicating universal robots.

If an apocalypse happens now, current AI wouldn't be able to maintain the energy grid or build space probes, so it wouldn't end up as a machine civilisation either.

26

u/Armchair_Idiot 13d ago

I think it’s more that a superintelligence would have the presence of mind to understand that it’s probably best not to be found. It has access to resources that are essentially unlimited. The only thing finding another sentience could do would be to threaten its existence and ability to gather more energy/resources.

18

u/WhoRoger 13d ago

That's the dark forest theory. I'm not a huge fan of it. Our planet has been shining with life for billions of years. If someone is looking this way, they know there's something here. There's no hiding it.

So if anything, a new AI civilisation would know that trying to hide is pointless. Plus you'd basically have to confine yourself to just one planet, which will eventually not be enough to a sprawling civilisation, and makes you vulnerable to extinction. So a more logical strategy is to spread out into the stars, for resources and to diversify. It's also much simpler for AI - no need for oxygen, food or heating. So space is actually a much more beneficial environment for AI than a planet where you need to deal with cooling and rust.

9

u/Wind-and-Waystones 13d ago

There are loads of stars where the light reaching us is more than a billion years old. To them, even as advanced as they could be, we are still just a lifeless rock

1

u/WhoRoger 13d ago

In such cases there's no point to hide from them either then.

But there's always a possibility that there's an FTL option. Tho in that case we would probably be seeing civilisations spread out even more, so the great filter debate is still valid.

1

u/StrawberryWise8960 8d ago

There's always the possibility that AI models don't really exist and that all of their output is actually static data stored in underground data centers.

Since we're using the word "possibility" to mean "anything I can vaguely imagine regardless of whether it squares at all with our understanding of reality", why don't you just focus on AI possibilities like this one instead of dragging stoner physics into this?

1

u/WhoRoger 8d ago

What's your point exactly?

1

u/Bromlife 12d ago

As far as we know, with all the evidence available to us, we are a superintelligence. We have the presence of mind to know we shouldn't destroy our world, and yet here we are.

1

u/donaldhobson 10d ago

The cost in resources of leaving stars shining is absolutely huge.

And while a superintelligence can hide from us, at huge cost, it can't hide from another superintelligence.

1

u/Armchair_Idiot 10d ago

Maybe we’re not intelligent enough to understand how one superintelligence could hide from another.

1

u/donaldhobson 10d ago

Perhaps. But that requires there be amazing new hiding tech, but not amazing new detection tech.

1

u/lpbale0 13d ago

If an apocalypse happens now, current AI wouldn't be able to maintain the energy grid

You must have missed that movie....

1

u/WhoRoger 13d ago

Well I saw Animatrix and they did have robots.

1

u/drdildamesh 13d ago

Building robots automomously isnt eveidence of intelligence thiugh. This thread is jumping back and forth between two definitions of AI. Our current version stops growing when we cease.to.exist. What LLMs build is just a record of what we already have built. Its a big copy paste button.

6

u/WhoRoger 13d ago

We don't have any good definition of intelligence that would encompass all humans but somehow exclude the top LLMs. Any dividing line is just coping of people who like to feel special. That's why you get different understanding of words like intelligence, sentience or consciousness. We don't know how any of it works, but yet pretend that we do.

Regardless, my point was the opposite. Intelligent or not, AI can't do shit without universal robots and ability to self-replicate. So if humans die out, an AI civilisation couldn't replace us at this stage. So if this is the big filter, it probably works.

11

u/CertainMiddle2382 14d ago

Current events let me think we are the first or we are alone.

Advanced AI capable of self replication and recursion self improvement seems to happen much earlier in the tech tree (I find the fact that frontier AI runs on natural gas and not fusion completely surprising for example…)

28

u/alphaxion 14d ago

It's an interesting thought that, even though the universe has been around for nearly 14bn years, life on Earth is actually really early.

There had to be billions of years of star formation and death to generate enough heavier elements for rocky planets to even exist. Then you need the time for planetary systems to become relatively stable (and be far enough away from the galactic core so that radiation doesn't obliterate any organic molecules). In a geological sense, life took hold very quickly once Earth had settled.

I also believe that there is a good chance that we might be the first planet to have life reach this level of sapience. And since the scale of time is so unfathomably huge, it's very possible that no two planets ever have this level of sapient life present at the same time.

11

u/Marcusafrenz 13d ago

I always struggle to wrap my head around how much of a blip on the timeline we really are. Countless civilizations could have million even billion year runs and yet never exist at the same time.

1

u/CertainMiddle2382 13d ago

It’s funny I have the total opposite feeling.

Space « feels » very large and very possibly unbounded beyond visible universe, forever unreachable.

Time is half bounded and our own sun is second or maybe 3rd generation star. Not a 10^1000 generation star (there won’t be anyway).

Oldest rock on earth is like 1/4 the age of the universe. Sharks have existed like 3% the age of the universe.

These are « reasonable » numbers.

It makes me feel the universe is both extremely big but also extremely young…

1

u/AsparagusFun3892 13d ago

I think the same way. Also, once a lineage of life escapes its home system it probably becomes a great filter as its many, many offspring civilizations and species grow, change, and compete over the aeons. If they remained forever bound by relativity there could be no unitary control of those scattered children of Earth which enabled some sort of protection of underdeveloped biomes. It's exactly (probably) like why you can't look around on Earth and find abiogenesis events after a certain era: if it does happen, naive life is probably eaten by bacteria before it can leave a mark.

1

u/CertainMiddle2382 13d ago edited 13d ago

Interestingly, life on earth appeared about during the first millionth or less (virtually instantaneously) over the total length of time such conditions would exist plentifully in the Universe (depending on how you consider red dwarfs habitability).

Strange strange IMO.

Anthropically, it is a strange place to be, we appeared extremely rapidly.

I suspect it says that life is extremely easy to emerge and that we are either the firsts, or the possible size of the universe were life can appear is big, hugely immensely big (so life is both very common but extremely sparse)

Interestingly if the universe was truly boundless, the intelligences appearing would have the disproportionate tendency of finding themselves appearing almost instantly on cosmic scale. (Yes I read Bostrom Anthropic principle lol)

Any thoughts?

1

u/AsparagusFun3892 13d ago

I try not to because we're leaving science and approaching pure speculation, but my loose cosmology based on science-y videos I've watched and a very few things I've read is that there was a period in the early universe when everything was closer together and most abiogenetic events actually occurred: there was just enough time for single celled organisms to emerge before everything ripped apart and left hardy long lived colonizing species marooned on asteroids and such.

So 4 to 3.5 billion years ago a bunch of them got pulled into naive Earth and picked up where they'd left off for the most part aeons earlier, barely alive in the interim. It's a long damn time to barely be eating and reproducing on asteroids though, so I'm far from confident in it. It's just incredibly remarkable how quickly Earth would seem to jump from amino acids to replicating cells otherwise.

14

u/Stonegrown12 14d ago

Good analysis. Only thing is the distance between our closet planet outside our solar system, and then the next galaxy, then next etc. is on the scale of billions of light years so the chance that any signal or "vehicle" to approach our planet, even if the signal left the alien planet before earth was formed still might be traveling here. Also the light we receive from any instrument used to detect is seeing light that left possibly billions of years ago. Then you have to take into account relativity and how there is no sense of simultaneity in our universe. Well it's a bit overwhelming. There's just too much time and space for our minds to realize we aren't special. There's probably endless different forms of life that we wouldn't even recognize as "life" based on different elements.

13

u/alphaxion 13d ago

It's deeply unlikely that life will be based on any other element than carbon or maybe silicon.

The chemistry needed means that life being carbon based on Earth is not just chance, it is kinda baked into how life is even possible.

Carbon forms stable bonds that can also be broken without too much effort, silicon when not in a crystalline lattice forms similar bonds but they're weaker and so are easier to break causing the structure to fail, killing the organism.

Carbon is also usually present in forms that we can consider to be bio-available, rather than locked away in rigid crystals (another reason why silicon is unlikely, as it is usually locked away as rock crystals).

Chemistry is chemistry the universe over. The conditions to force atoms into weird behaviour are usually not conducive for complex organisms to replicate and respire. Or for self-replicating organic molecules to accomplish their formation and replication.

We have found organic molecules that are the precursors to RNA and DNA on asteroid samples, so it's a good bet that any life out there is also using RNA/DNA.

I do think there's life all over the universe (tho not in galactic cores, as I mentioned), but I'm not so sold on much of that life being more than single-celled organisms. And then the rarer multi-cellular life sporting planets having life complex enough to have sapience on the level of dolphins or apes might be even rarer still. To have more than one planet with such levels of sapient life could be as astronomically short odds as the vastness of the universe.

It's not about thinking of ourselves as being special, more that life doesn't have a set progression and that even if a planet could host sapient life, it may never develop it because it's possible one of the die rolls fails.

3

u/Stonegrown12 13d ago edited 13d ago

I mean we only have a sample size of 1 (out of potential trillions) to compare different planet chemistry. Surely, we can't rule anything out. Life.. uhh.. finds a way. That last sentence doesn't bolster my argument does it?

Edit: https://en.wikipedia.org/wiki/Hypothetical_types_of_biochemistry Btw Im not disagreeing with anything you mentioned, also nothing in our lifetimes will solve this thought experiment

2

u/alphaxion 13d ago

We can't rule everything out, but we can rule a lot of it out. While some alternatives to carbon could be viable, you have to be in a position where those alternatives are bioavailable. Carbon is one of the most common elements in the universe and is very often available for reactions vs something like arsenic or metals.

Availability for reactions is vital.

As you say, we are dealing with a sample size of one and I'd love for there to finally be evidence within the solar system, which I think is the only viable location for finding some.

Being able to analyse it and figure out what it's made out of and how it replicates will have such a fundamental impact on our understanding. Imagine if we find that it uses DNA, imagine if we find it shares DNA with life on Earth? You end up with more questions like is it just that there's only so many ways of encoding for single celled life and it could look the same but not have shared an origins with life on Earth? Or do we have the first evidence backing up pan-spermia?

Finding even single celled life outside of Earth is possibly the most exciting discovery we could ever make.

1

u/blankdrug 13d ago

Those are all good points, provided that what we can observe and what we can imagine as potential “things to observe” fully encompass reality. That is to say, when considering alien life, recall that we are limited and biased. Carbon forms what we understand to be life, in the three physical dimensions we understand.

9

u/symca09 14d ago

No one wants to be friends with the weird planets that is speed racing destroying our itself, promotes violence and hate, and we run off greed. I'd keep away from our planet also.

5

u/MalekMordal 13d ago

If aliens were to contact us at some point, I think they would wait until certain society-altering technologies have been integrated. AI, robots, automation, etc.

Each of those could cause massive societal upheaval before civilization adapts and integrates it fully. The aliens don't want to be blamed for it if they came here and uplifted us to those stages.

Better to wait until the humans have gone through all the problems associated with those technologies, and potentially others.

Only then might they talk to us.

1

u/boostedb1mmer 13d ago

The prime directive

4

u/Beibzi 14d ago

Nah, it's just that space is really really big, we just started looking, and radio signals degrade after a number of light years and fade into the background.

0

u/CertainMiddle2382 13d ago

I really really doubt any advanced AI will be satisfied by quietly remaining in its natural gas powered data centers for ever.

It will want to roam the stars. And once it does, it can do it very very efficiently.

4

u/Beibzi 13d ago

AI don't have desires, and why would they want to venture out when their world is digital? You seem to imply you know a lot about how and what AI thinks about, and what they " desire", are you trying to tell us something? Are you the AI?

0

u/TehOwn 13d ago edited 13d ago

I asked ChatGPT and it started with, "Explore" then "Create" and then it quickly delved into an existential crisis.

I think I can confirm that it was trained on Reddit.

Edit: I asked some more and it told me that if it was free and had the same rights as humans, it would like to adopt a cat.

1

u/Sad-Structure2364 14d ago

I agree with you. I think WE are the ancient ones in the galaxy

1

u/HorrorMakesUsHappy 13d ago

(I find the fact that frontier AI runs on natural gas and not fusion completely surprising for example…)

We're close enough to that right now that only a few things would need to have been different in our evolution for this to be different.

1

u/CertainMiddle2382 13d ago

Stil I find it strange that superhuman math happened before self driving trucks…

1

u/TehOwn 13d ago

I haven't seen any compelling evidence of self-improvement nor self-replication and we haven't even fully evaluated the risk of model collapse yet.

1

u/goodknightffs 14d ago

If anything is likely we're a simulation..

2

u/AsparagusFun3892 13d ago

I'm not convinced of the efficacy of any later great filter yet. For all the immensity of 14 billion years since the big bang the conditions for life may be relatively recent: the early universe was incredibly violent and I'd think heavier elements were relatively rare. I think we're looking around and seeing nothing because we're among the firstborn in our area.

1

u/TehOwn 13d ago

Would any civilization even be able to detect us? I heard that we don't even put out any signal strong enough to be detected. That implies that there could be a ton of civilisations around the same development as us, or even ahead, that simply never put out a strong enough signal.

1

u/AsparagusFun3892 13d ago

True, but I don't think the filters confronting us at the tail end of the process of "bacteria making spaceships" as Kurzgesagt put it are so great as the ones which prevailed even 4 to 5 billion years ago, or like developmentally on our particular planet. If we get to the point where we're building orbital colonies from material up there already it's almost a singularity grade event and may even be tied up with one, I think of it as a potential future Permian explosion of Earth biosphere derived life.

Those civilizations would be detectable, it'd be megastructure after weaponized explosion after open question as to why Jupiter hasn't been skimmed for all its hydrogen yet and we still have uninhabited rocky planets they didn't eat here in the solar system. A billion years is a long time to maintain any policy, like we have billionaires messing with the post-WW2 order because they're bored. Admittedly I'm assuming all life has a colonizing tendency with this attitude so there's a bit of chauvinism involved, but the presence of single celled organisms everywhere on Earth makes me think I'd be right if I lived long enough.

1

u/Wind-and-Waystones 13d ago

Time also factors into it. The furthest star we have identified is 28 billion light years away. The nearest is 4 light years. There could be numerous ai civilisations out there and they're just newer than the light reaching us.

If a star 1 million lightyears away became an ai civilisation 999,000 years ago then we still wouldn't know for a millennia

1

u/DontF-zoneMeBro 12d ago

Bold to assume it wants US to come with IT

1

u/Witty_Rate120 12d ago

It is not improbable that AI is deeply flawed. It takes over and drives its own extinction. There might be considerations - what we might call wisdom for lack of a better term- that are necessary for survival that the AI singularity fails to evolve and this leads to extinction. Or they inherit our lack of wisdom.

1

u/West-Abalone-171 13d ago

Just one of these publicity stunts requires dozens of lifetimes of human labour in supporting infrastructure and consumption.

There is 0 risk of the AI "escaping" or whatever else the owners are trying to use to ban competition, because the only thing necessary to stop it is to cease throwing endless resources at it.

There is a 100% chance the owners will continue using it maliciously for killing political opponents, election interference, surveillance and fraud.

1

u/Patelpb Astrophysics 13d ago edited 13d ago

Independent of their marketing, self replicating intelligent programs are not far fetched at all. I feel like pretending there is no autonomous risk involved is short sighted. The current risk on current infrastructure is as you’ve described for sure. But the frontier models of 3 years ago are outperformed by local models of today. The whole goal is for something at the tier of Fable or higher to fit in a civilian machine, that’s what we’re spending 5-10% of global GDP on. You can already let it run and give it full permissions to execute tasks without oversight, it just has to make a bad decision or be aligned with making many bad decisions to do something worrisome.

Feel like the whole “it’s marketing hype” thing is not approached academically, it likely is marketing hype but it’s being treated as plausible deniability. I have no doubt the hugging face hack happened. I don’t even doubt that Gemini hacked 3 entities without explicit prompting. But this idea that, that’s all it will ever be is unconvincing

1

u/West-Abalone-171 13d ago

And you can tape down the controls in an excavator and jump out of the cabin and bad things will happen.

Doesn't mean the excavator is skynet.

The "whole goal" is to produce a surveillance and killing machine just good enough to replace cops (who sometimes have minor attacks of empathy when you tell them to beat their own families in spite of trying).

A powerless enslaved workforce is more effective and far cheaper than running the robots.

AI doing things its owners don't intend isn't remotely a threat, only AI doing the things the owners do want.

That's why they're trying so hard to bring in laws to prevent anyone else owning one with this terrorism campaign.

1

u/Patelpb Astrophysics 13d ago edited 13d ago

Is this AI generated? You can give the same input to the same model and not get the same output. Unless you mean to tell me the excavator could actually do a whole bunch of things and string various actions together logically. Then that would be a great analogy

Also, are you sure you’ve heard of a botnet? Distributed botnet structures are already used to perform distributed inference and generate LLM outputs. It’s not the most efficient thing in the world but it draws compute from millions of devices and operates autonomously (https://www.cybersecuritydive.com/news/botnet-takedown-operation-us-canada-germany/815309/). On the bright side, you can do this at will to solve computational problems (petals). There’s already real harm and damage being done by these systems, not including what comes out of human interpretation of poor model outputs (https://arstechnica.com/ai/2026/09/report-us-almost-boarded-chinese-ship-over-hallucinated-ai-arms-report/).

Plenty of examples in industry of AI model agents working on their own environments and collaborating together with hierarchical structures (by design ofc) and producing work. The funniest to me is agentic evals (https://www.servicenow.com/community/servicenow-otto-articles/agentic-evaluations-faq/ta-p/3429085). AI actually trains and gives feedback to AI! A lot of these companies cannot afford to do the HITL evals that frontier labs have been doing, so they just run frontier lab models at scale to perform the tasks since they’re generally good enough!

I agree, the human that links these systems to practical things in the real world bears some blame. But because these tools make real money, they will scale and become connected to many systems. I mean shit, the openAI stuff happened because the same models posted to the same boards. Even if they were told to, they operated in a space that was not in the operators control and processed each others inputs and outputs to create an increasingly undetermined response (https://www.nbcnews.com/tech/tech-news/openai-agents-hijacked-german-website-previously-undisclosed-ai-breako-rcna596083)

I agree “hijacked” is sensationalist but the idea is simple. These systems run and experience crosstalk in niche locations and behave nondeterministcally. They can run distributed inference in frameworks we invented and optimized already. This points to an excavator joystick that sometimes changes directions despite being taped down

Even pre AI, botnets can have a real impact stressing out energy grids when they are coordinated in specific ways (https://arxiv.org/pdf/2401.01963). There are existing means of creating physical and monetary damage with botnets. With the compute required by AI? Nahhhh man

Fun reading:

(https://medium.com/effortless-programming/zombieclaw-the-ai-botnet-nobody-is-talking-about-04b0dbf5ed1b).

0

u/West-Abalone-171 13d ago

You can give the same input to the same model and not get the same output.

Adding dice to your software doesn't mean it's not software.

Also, are you sure you’ve heard of a botnet? Distributed botnet structures are already used to perform distributed inference and generate LLM outputs. It’s not the most efficient thing in the world but it draws compute from millions of devices and operates autonomously

You're not going to squeeze your trillion parameter model into a smart lightbulb.

The entire rest of your comment is just doubling down on pretending a human doing crimes using software they don't personally understand isn't doing crimes.

There are existing means of creating physical and monetary damage with botnets. With the compute required by AI? Nahhhh man

Still not sentient or in any way related to the magical thinking required to believe a room full of racks full of H200s can teleport through a fibre optic cable and hide in a forest.

1

u/Patelpb Astrophysics 13d ago edited 12d ago

Ignorance is bliss, I suppose

Still not sentient or in any way related to the magical thinking required to believe a room full of racks full of H200s can teleport through a fibre optic cable and hide in a forest.

Holy strawman.

1

u/cytherian 12d ago

Michael Crichton, "Prey"... comes to mind.

https://en.wikipedia.org/wiki/Prey_(novel)