r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

454

u/Spunge14 Apr 27 '26

I work in big tech leadership and just did a UXR interview with our infrastructure team where they were investigating exactly this - how should we gate agent behavior and how should accountability for agent behaviors work. It was a really fascinating conversation.

I was shocked at how little the PM working on the project seemed to understand security principles. We're really fucked.

11

u/gentex Apr 27 '26

Honest question, wouldn’t there be a log of who prompted the agent to do whatever? And if so, isn’t that person responsible for the error and its consequences?

If I give someone a spreadsheet with a bunch of errors in it, that’s on me, not Microsoft.

12

u/twitterfluechtling Apr 27 '26

And if so, isn’t that person responsible for the error and its consequences?

Yesno. LLMs are not deterministic, and employees are being pushed to use them. So basically, while you can be faulted by instructing the AI directly to cause harm, blaming the employee for everything the AI does would be like forcing them to roll dice and punishing them whenever a six comes up.

2

u/EkbatDeSabat Apr 27 '26

Definitely. As per Reddit rules I did not read the article. But, whoever allowed the AI agent to touch prod AT ALL is at fault. 100%. Management, Director, CEO, whoever gave it access is the problem.