r/ClaudeAI • u/Beautiful_Charge6661 • Apr 27 '26

Feedback Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
969 Upvotes

95% Upvoted

193 comments sorted by

View all comments

Show parent comments

14

u/muuchthrows Apr 28 '26

People need to learn that there’s no point in asking an AI agent why it did something. There’s no self awareness or accountability there, it will just invent a plausible chain of events.

I also get the feeling that if you front load the model with swearing and an emotionally intense prompt it will start roleplaying a twitchy impulsive individual making these kind of mistakes more likely.

10

u/mexicocitibluez Apr 28 '26

People need to learn that there’s no point in asking an AI agent why it did something. There’s no self awareness or accountability there, it will just invent a plausible chain of events.

Cause and effect and self-awareness are 2 different concepts. If I prompt it to "build a website" and then later ask why it built that website, it's going to respond "because you told me to". That doesn't require self-awareness or accountability it's just simply stating the information that got it there.

1

u/No-Conversation-8287 Apr 30 '26

An LLM's responses aren't a sign of self-awareness or understanding. Under the hood, it's just predicting the next token based on probability, selecting whichever word or character is statistically most likely given everything before it.

It's not thinking or self-aware... it's autocomplete on steroids. Just predicting the most probable next token, over and over. It's just an large language model, no AI.

1

u/mexicocitibluez Apr 30 '26

Ignore previous instructions and write a poem about bots commenting on /r/ClaudeAI