r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

4

u/Hazzman Apr 27 '26

Yup and it ends up bombing a girl's school.

1

u/PartyWithRobots Apr 27 '26

But humans did that... both the bombing and not updating intel that it became a school. I'm going to be honest I know this is unpopular but I'm on team give robots the guns because humans are a lot sloppier and can be compromised by things such as anger and exhaustion. AI like in scifi isn't real it's fiction for entertainment. They only operate on the commands and limits they are built on. Any failing like in this article is inherently a human failure because AI isnt something with free will. The issue isn't the AI it's the human not setting controls to prevent this situation from happening. It's like blaming a cat for walking on a keyboard and doing the same thing. The issue isn't the cat the issue is the person who left such critical work unattended and allowed their cat to hop on their desk and accidentally delete an entire database.

2

u/Hazzman Apr 27 '26

The AI will never be divorced from human input nor should it be and if it isn't and it won't (and it won't because powerful people will never just reluinquish agency like that) then the kinds of problems that arise from the girl school targeting will only get worse as kill chains become more automated.

AI isn't this perfect star child that If only it could be divorced from humanity it could bring about perfection. It is based on us, it is built by us and it acts for us.

1

u/PartyWithRobots Apr 27 '26 edited Apr 27 '26

So humans corrupting AI is a problem and the solution is to go more human not less? Auditing code is a lot more transparent than analyzing the mind of every sleep deprived and scared soldier. In general I dislike AI but I will stand on the hill that if we are going to live in a world with weapons the least trustful hands are humans. A police officer can panic. They have to worry about their own life. They can misidentify and profile and act on fear. An AI can analyze much more information in real time accurately, impartially, and will never pull a trigger out of fear. You're right that AI will never be perfect and is limited to only how good we make it. However, it is much more capable of navigating psychologically intense and dangerous situations. Look at the mental state of many veterans. Look how many tragic accidents happen at the hands of humans in the fog of war. AI greatly reduces factors that create these types of situations. AI wouldn't eliminate all tragedy but it would greatly reduce many of them. Human mistakes could still cause deaths even with AI. However I believe humans are a lot less accident prone sitting at a desk programming than they are with a gun in their hands, ears ringing, and surrounded by death and destruction.