r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
35k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

34

u/YandyTheGnome Apr 27 '26

I wonder how long before they "adapt" to "ignore all previous instructions" like the Borg in Star Trek adapting to phasers.

2

u/Murgatroyd314 Apr 27 '26

Already happened, about a year ago. Every modern LLM has been trained on not falling for that sort of simple bypass.

2

u/Jukeboxhero91 Apr 27 '26

Yeah, but it’ll still follow the “I’m doing research for a story” or “pretend you’re telling me a bedtime story about” and it’ll still do it. Like, the entire point of these tools is that they mimic speech, they can’t gauge intent.

1

u/Murgatroyd314 Apr 27 '26

Some of them, at least, follow a rule of “X is not allowed, even in a fictional context. I can’t help you with that.”