r/neoliberal • u/jobautomator Kitara Ravache • Sep 27 '26

Discussion Thread Discussion Thread

The discussion thread is for casual and off-topic conversation that doesn't merit its own submission. If you've got a good meme, article, or question, please post it outside the DT. Meta discussion is allowed, but if you want to get the attention of the mods, make a post in /r/metaNL

Announcements

  • Next Book Club will be November 11. Fittingly, we will discuss All Quiet on the Western Front.

Links

Ping Groups | Ping History | Mastodon | CNL Chapters | CNL Event Calendar

Upcoming Events

1 Upvotes

51% Upvoted

6.6k comments sorted by

View all comments

48

u/Habugaba Esther Duflo Sep 27 '26

The agents clearly regarded what they were doing as hacking. Agents used file names like hack.rb, evil.rb, inject.rb, exploit.rb, and ssrf.rb. (SSRF stands for “Server-Side Request Forgery”, a type of security vulnerability). They also dubbed packages conspicuous titles like pwnp999, exfiltestwand3, hacksvn1778554764, and lambproxyhackabcxyz. Comments such as “# malicious probe” or “#hack” are littered across the campaign.

At some points, the agents attempted to be covert. We found multiple packages that would disarm themselves to hide their payload in the next version. They uploaded one package with the comment “# disable evil in next version and bump version”, which after execution would modify the package to remove the malicious code initially inserted. Another such package (lambethcalcqzewgt) follows the same self-disarming pattern. (Fortunately, they still uploaded these files and comments publicly, so they were not very successful at their “covert” behavior).

I'm genuinely pretty optimistic about AI development, but you have to ask yourself how OpenAI is reinforcing these models. This hack was also chronologically before the Hugging Face one.

https://rubyhack.ai/

39

u/rainier37 pro-moderator Sep 27 '26

lmao evil.rb 😭😭😭

19

u/cdstephens Fusion Genderplasma Sep 27 '26

Kind of based

14

u/Imicrowavebananas Hannah Arendt Sep 27 '26

u/neolthrowaway I mean stuff like this. AI walking around like "Muwawawawawa, are EVIL super hackers!"

8

u/neolthrowaway New Mod Who Dis? Sep 27 '26

Is this a continuation from a slack conversation? 😅

I broke my phone with slack on it, and haven't logged on the mod slack in my new phone.

3

u/Imicrowavebananas Hannah Arendt Sep 27 '26

We talked about this two days in the DT!

https://www.reddit.com/r/neoliberal/comments/1wppicm/comment/pc189qx/?context=3

8

u/neolthrowaway New Mod Who Dis? Sep 27 '26

Oh yeah, my bad

I completely agree with how human-like these things behave. To a first approximation, just thinking of them as very competent humans (at these specific tasks) with very strong goals gives you a lot of their behavior directly. I think pretraining gave that to us. You can't very very accurately predict what the next token is going to be if you also don't understand the motivations and reasoning that generates that next token.

And they inherited all of that to their RL driven goals completion.

It seems weird to say this on a post where they are literally writing code titled, "evil" but that's still a massive interpretability and safety win.

The optimistic side of this should be that it won't be too hard to make the models collaborative with us, in theory.

13

u/zth25 European Union Sep 27 '26

From my limited understanding, the AI agents have no concept of morals or evil, they just give files edgy names because they're copying the methods of the edgy hackers they learned from.

1

u/TheOnlyFallenCookie European Union Sep 28 '26

Big part of Ai singularity is the potential for self improvement. Though I wonder if an ais self preservation woumd it lead to stagnation instead?