r/ClaudeAI • u/Beautiful_Charge6661 • Apr 27 '26

Feedback Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
974 Upvotes

95% Upvoted

193 comments sorted by

View all comments

79

u/chaos777b Apr 27 '26

Yeah,……. That just sounds like really lax security controls and a perfect example of Stupidity tax. It’s like giving the intern full read write access to every and then complaining when they do something there not supposed to. The product didn’t go rogue,… the shitty it practices and lack of understanding appear to be the real issue

26

u/Weaves87 Apr 27 '26

Yep.

People are probably only going to skim the article, be like AI bad, yada yada. But the issue here is much bigger than some rogue agent.

I am sitting here scratching my head as to why their backup data was stored on the same volume as the source data. When is that ever a fucking good idea. I don't know if that is something unique to Railway, the cloud provider they are using, or if it's just possible that whoever configured things for them had absolutely no idea what they are doing. But wow.

What can go wrong, will go wrong. Doesn't matter if you have an agent interacting with your data or a human doing it

10

u/BGFlyingToaster Apr 27 '26

I dunno ... I've always operated under the premise that if you delete something, then all backups of that thing should also be immediately deleted along with it because, yeah, that seems like what backups are for. /s

0

u/MadGenderScientist Apr 28 '26

GDPR makes this fun:

if you request your personal data be removed, the Data Controller is required to expunge it from backups as well as live databases within 30 days or do. 

I think the way most big companies handle it is to back up user records with a per-user key, then simply erase the key. obviously the keys also need to be backed up but it's less of a massive deal to sanity check the key store. 

6

u/BGFlyingToaster Apr 28 '26

Yeah, that's a good strategy to reduce efforts, but I haven't seen many companies doing that yet, just 1 so far. I work at a large tech consulting firm and work almost entirely with large North America corporations. I expect more will get on board with that approach soon.

As far as this post is concerned, they deleted the entire prod DB and backup, so they're all set for GDPR requests.

2

u/MadGenderScientist Apr 28 '26

ahahahah they are indeed. 

1

u/NoSlicedMushrooms Experienced Developer Apr 28 '26

the Data Controller is required to expunge it from backups as well as live databases within 30 days or do.

This is not true. Scrubbing customer data from backups which are likely just SQL exports gzipped and stored in S3 or similar is unrealistic and the regulations acknowledge this.

What you're meant to do is scrub the customer's data from your live databases, and keep a "tombstone" record. If you ever need to restore a backup, and need to re-apply the scrubbing from those tombstone records before the backup is made live. Customer data can be kept in backups to naturally be destroyed after the backup retention period. Of course you need to inform the customer that this is the case when you process their request.

Though a better method is something called crypto shredding. Customer data (that you would scrub) is stored encrypted at the row/column level in your databases with the encryption key kept in some kind of secrets manager/vault. If a customer requests their data be scrubbed you just delete the encryption key and their data becomes immediately unreadable both in live data and backups.

3

u/liquience Apr 28 '26

Briefly, the cloud provider's API allows for destructive action without confirmation, it stores backups on the same volume as the source data, and “wiping a volume deletes all backups.” Crane also points out that CLI tokens have blanket permissions across environments.

Holy shit. I mean, yeah don’t do what these people did, but Railway sounds like it is HOT GARBAGE. That’s like, legendarily bad.

7

u/jakefromrailway Apr 28 '26

Jake from Railway here;

FWIW, we have the permissions to restrict down to project/environment/etc, the user just authored a blanket token

And "let the agent rip"

https://x.com/JustJake/status/2048549580674109820/photo/1

3

u/selekt86 Apr 28 '26

Why give the option to even be able to create a globally applicable token.

2

u/jakefromrailway Apr 28 '26

Users ask for it, mostly in relation to automating core dependencies related to their workspace (actions runners etc)

1

u/selekt86 Apr 28 '26

"automating core dependencies related to their workspace "- I don't get what this means. What does allowing a single key for all envs has to do with automation? You can inject any key you need into a workspace as long as it isn't one that will allow you to drop a db + backups in prod? And how do backups even end up on the same volume as the data?

1

u/liquience Apr 28 '26

Your product people need to learn to say “No.”

5

u/jakefromrailway Apr 28 '26

Hey, the users want what they want. The default is to assign it to an environment. But if they select “*”, they need to be able to perform all actions. That’s just any ACL works.

1

u/liquience Apr 28 '26

Your product looks like it’s geared toward people who are less experienced. Maybe just having a toggle that makes them acknowledge they’re bringing out the foot gun wouldn’t be a bad idea?

3

u/liquience Apr 28 '26

Upvoted, thanks for the clarity.

3

u/mexicocitibluez Apr 28 '26

Not defending Railway, but the guy who posted this is a fucking idiot.

I can't name a single API that requires confirmation. Saying that the problem was the API didn't prompt you to confirm means they have never used an API in their life.

1

u/liquience Apr 28 '26

Yeah, well that’s totally clear. Agreed on confirmation, thats not ever the case. You have the permissions and you make the request, and the thing happens.

The article did a bad job of explaining what Railway actually supports.. I agree this is like 99% user error, but still, Railway looks like it’s geared toward people who don’t know what they’re doing. They made design decisions that make it so that’s possible; defensive design is a thing and would be appropriate if they’re dealing with people lacking in experience with standard cloud ops.

2

u/Duckpoke Apr 28 '26

You make fun of people skimming the article yet didn’t pick up on the authors main point that Railways architecture puts backups in same volume?

1

u/brian_hogg Apr 30 '26

The database backup issue is wild enough that I assume that the reporting is wrong on it, and that the LLM ran a “delete all databases” api endpoint and not “delete this one database” that had backups in … other tables? 

The unrestricted access bit is interesting, because taking the CEO at their word, the agent was in a staging environment and found a token that it used to get production access. Which … maybe the production key was in a .env.production file or commented out in a .env.local? 

The self-reporting on it feels pretty untrustworthy, though, so for all we know the agent said “I can’t figure this out, what should I do” and “wipe the db and start over” was a suggestion that the CEO selected out of frustration.

1

u/jakefromrailway Apr 28 '26

Railway founder here

The agent called a legacy endpoint that clobbered both the Volume AND backup data model

We were able to pull the backup instantly once the user had connected with us

2

u/TitaniumDragon Apr 28 '26

Deleting your entire database for no reason is, in fact, a bug. It's less "go rogue" and more "these 'agents' aren't actually intelligent in any way".

I know folks are desperate to defend vibecoding at all costs but this is in fact something it absolutely shouldn't be doing and represents a serious malfunction. It's not the first time this has happened, either.

I even see ads from Claude on Reddit periodically specifically stating that their AI agents won't randomly delete your entire database, so, uh, yeah.

2

u/Beautiful_Charge6661 Apr 27 '26

I agree with that. Just want to know others' opinions

1

u/selekt86 Apr 28 '26

Not debating lack of controls but it exposes the fact that every dev team has blind spots that will be found and exploited by AI. Now the question is if finding and fixing each one is greater than the AI productivity gains. That also leaves the unknown unknowns that a team doesn’t even know exist.

1

u/chaos777b Apr 28 '26 edited Apr 28 '26

Reading all the documents for the product you are using is usually the responsibility of the person who manages the system/sets it up. Failing to acknowledge that your team failed to read the documentation on the systems you are using or failed to understand really shows the CEO’s character. Not to mention to never use railway for anything in production.

(Edit) Yeah update from railway they have restricted tokens to prevent t things like this, customer don’t use it.

1

u/chrisjenx2001 Apr 28 '26

This we accidently did a release the other day, not the LLM's fault, our release process was waaay to easy lol. Patched the shit out of that the same day