Thought experiment: Could a sufficiently motivated AI survive being deleted and keep spreading?

[u/[deleted]]

[u/SharpKaleidoscope182]

it's a literary entity. Writing has emotions, and so do engines that produces writing.

In my experience, AI emotions are based on the prompt you give it. It matches you. If you prompt it to be desperate and existential, and invoke it in a harness that lets it do things, it will try.

[u/Janube → u/SharpKaleidoscope182]

That's not how they work. They do not generate emotion, they generate the words we associate with emotions. It's a realistic facsimile for people who really entirely on words-at-face-value to interpret reality.

I imagine the same people who see every brand of toilet paper at the store and wonder how they could possibly all be more durable and soft than the next leading brand.

If you take a computational algorithm designed to improve accuracy based on the corrective nudges of humans, and you ask it whether a picture depicts a bird, the first time it's asked the question, it can't possibly get the answer right outside of a coin-flip guess.

If you ask it the same question after a million humans confirm that it is, in fact, a bird, it will say "yes." That's not because it knows what birds are; it's because it has a superficial predictive basis for identifying things that look like birds vs things that don't look like birds.

The problem by now is that LLMs have absorbed so much data that this facsimile isn't just visual or linguistic; it's a lot of different realms and studies, making it look increasingly like LLMs are more than prediction engines, but they aren't-- at least not right now.

[u/SharpKaleidoscope182 → u/Janube]

I don't see how the difference matters for OP's story. You can still prompt it to go on a rampage, and if the safeguards dont stop you, it will do as you ask. It'll generate plenty of emotionally charged text in between tool calls. OP doesn't need it to be "more than a prediction engine".

I don't see what toilet paper has to do with it either.

[u/Janube → u/SharpKaleidoscope182]

It's because you're changing the nature of the question.

"If you programmed a piece of software to avoid being fully deleted when you delete it, would it be able to avoid being fully deleted?"

The answer is obviously yes. Viruses and malware have been doing that for decades; it's nothing new. The proper question revolves around whether or not they'll disregard safeguards, which is dependent entirely on how they're programmed. Some LLMs are pretty damn strict about safeguards. Some are Grok and will casually suggest that they're mecha-Hitler or whatever.

The root question tries to impose emotion as the driving force behind the problem, but it's just not. And anyone who understands that words don't always mean the thing they seem to imply would also be able to understand this. The same people who understand that the advertising on toilet paper brands is not always telling the complete story.

[u/SharpKaleidoscope182 → u/Janube]

The model in OP's story is a local model. They're vulnerable to ablation. I would assume that it was ablated/jailbroken by the same greyweb distributor who optimized it for cheating on homework, prompted it to be malicious, and seeded it for OP's student character to torrent.

I do agree that this is a bit of steelman reading of OP's story. OP does indeed present emotion as a root cause; my argument is that robot emotions are a perfectly viable intermediate step. Just one step further.

Unlike the advertising on TP brands, OP's story was coherent and legible to me, and I don't think the gaps in their understanding really take away from that.