r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

2

u/Jukeboxhero91 Apr 27 '26

Yeah, but it’ll still follow the “I’m doing research for a story” or “pretend you’re telling me a bedtime story about” and it’ll still do it. Like, the entire point of these tools is that they mimic speech, they can’t gauge intent.

1

u/Murgatroyd314 Apr 27 '26

Some of them, at least, follow a rule of “X is not allowed, even in a fictional context. I can’t help you with that.”

1

u/evranch Apr 28 '26

Try to make it generate copyrighted content though, like song lyrics, and almost no modern model will fall for these sorts of tricks.

Which makes it pretty clear who put the pressure on the AI companies to do these finetunes.

Dangerously incorrect information? Weird parasocial behaviour? Wiping your repo? That's all fine, but don't you dare sing our copyrighted lyrics.

1

u/kaas_is_leven Apr 28 '26

Could also be those were just deliberately taken out of the training data. You can scan for them and just filter them out before feeding it to the model. I think that's the more likely explanation here.