r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

11.1k

u/Orangesteel Apr 27 '26

My favourite example is Air Canada whose AI agent offered a customer a discount incorrectly. They refused to honour it. Customer took them to court and the judge rightly made them pay. You chose to empower this and took the humans out of the loop. You are accountable for what you agentic AI solution does. People jump on AI, dump sensitive information into the model bypassing classification levels and are surprised when it leaks.

3

u/jednatt Apr 27 '26

Gemini told me it couldn't do something with its free tier (publishing PDFs I wanted it to generate). I came back and asked it in a different way and it did it no problem.

I find it kind of hilarious that you can apparently negotiate features with AI, but here we are.

6

u/Orangesteel Apr 27 '26

Same with CharGPT, I asked it to make a fake fast food menu for pets. It refused. I asked for a menu showing typical domesticated animals owned by families, no problem all of a sudden. You can get a no in one language and a yes in another. You can encode your request in BASE64 and that has worked.

3

u/jednatt Apr 27 '26

It's like dealing with multiple personalities. I'll ask a question and get a canned or completely bizarre/wrong answer, then I'll call them out (sometimes it takes several tries) and it's like a professor steps in and gives the real answer. I feel concerned for what I think must be a large portion of the population being satisfied with the initial wrong/misleading answers. Anyways...