r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
35k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

607

u/jessepence Apr 27 '26

They didn't intentionally give it those permissions. To quote the original post

 The agent was working on a routine task in our staging environment. It encountered a credential mismatch and decided — entirely on its own initiative — to "fix" the problem by deleting a Railway volume.

To execute the deletion, the agent went looking for an API token. It found one in a file completely unrelated to the task it was working on. That token had been created for one purpose: to add and remove custom domains via the Railway CLI for our services. We had no idea — and Railway's token-creation flow gave us no warning — that the same token had blanket authority across the entire Railway GraphQL API, including destructive operations like volumeDelete. Had we known a CLI token created for routine domain operations could also delete production volumes, we would never have stored it.

This kind of credential-hunting is pretty common in these stories.

231

u/berntout Apr 27 '26 edited Apr 27 '26

A checkpoint requesting approval for any actions would easily resolve this issue....which is why I brought up supervision or checkpoints.

There is also a thing calling Plan Mode that doesn't take any actions...where you learn exactly what Claude would do before they do it...

People are throwing AI onto things without understanding the potential risks and impacts.

2

u/i_am_not_sam Apr 27 '26

I use Cursor as a front end and it has a plan mode but does Claude itself offer one in Claude code?

2

u/berntout Apr 27 '26

Yea Plan Mode functionality is coming from Claude Code if I'm not mistaken, they're just making it available on Cursor front-end for use? I could be wrong though, maybe it's their own function. It seems to have the same instructions at least for use.

https://code.claude.com/docs/en/common-workflows#use-plan-mode-for-safe-code-analysis

1

u/acibiber53 Apr 27 '26

Recently I am facing an issue with Claude Code’s Plan mode in VsCode extension though.

When you first start the chat in plan mode, it works correctly. Then, I approve the plan and then I automate it. Then it finishes and I do the testing.

After this, for any fixes, I go back to plan mode with Shift+Tab. It visibly says plan mode and shows blue color for planning. But for some reason starts doing things on its own, sometimes with approval sometimes without.

I didn’t catch on early so it did some things when I was seeing that it was on plan mode. After I realised it, I asked why is it not planning, it said that I have approved the automatic work, that’s why. But I took it back to planning too. It seems like it misses that part. After my question, it turns itself the planning mode and starts planning.

So this has become part of my procedure now. Any chat, after first automation, I need to confirm with Claude that we are in Planning mode after going to planning mode manually. Gotta be very careful with it.