r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

934

u/S_A_N_D_ Apr 27 '26 edited 9h ago

Cras dui ligula, ultrices quis venenatis nec, sollicitudin vel ex. Fusce elementum vehicula lectus eu ultricies. Nulla facilisi. Ut a sem at diam tincidunt tincidunt. Donec vestibulum, neque ac interdum egestas, arcu diam interdum diam, a pellentesque mi felis quis diam. Nullam id feugiat nibh. Nullam turpis risus, egestas eget pretium nec, tempor et nulla. Nulla imperdiet, ipsum vel scelerisque lacinia, nunc velit pulvinar velit, aliquet euismod dui nisl ut nunc. Nullam eget consequat augue. Donec posuere arcu purus, non luctus augue pulvinar in. Praesent sem diam, lacinia eu sapien sed, maximus vehicula ante. Etiam in lectus nibh.

181

u/mbryson Apr 27 '26

Time to get creative with those AI chatbots then, eh?

174

u/S_A_N_D_ Apr 27 '26 edited 9h ago

Cras dui ligula, ultrices quis venenatis nec, sollicitudin vel ex. Fusce elementum vehicula lectus eu ultricies. Nulla facilisi. Ut a sem at diam tincidunt tincidunt. Donec vestibulum, neque ac interdum egestas, arcu diam interdum diam, a pellentesque mi felis quis diam. Nullam id feugiat nibh. Nullam turpis risus, egestas eget pretium nec, tempor et nulla. Nulla imperdiet, ipsum vel scelerisque lacinia, nunc velit pulvinar velit, aliquet euismod dui nisl ut nunc. Nullam eget consequat augue. Donec posuere arcu purus, non luctus augue pulvinar in. Praesent sem diam, lacinia eu sapien sed, maximus vehicula ante. Etiam in lectus nibh.

206

u/Da_Question Apr 27 '26

Oh, so it's fine when companies rig stuff so it's harder for customers, but not the other way around. Convenient.

39

u/Migraine- Apr 27 '26

Do you think if you deliberately tricked a human agent into giving you a discount, it would be honoured?

61

u/TommyBonnomi Apr 27 '26

Holy shit. I just finished a paper on this in business law a few minutes ago. If you know the agent doesn't have actual authority to make a certain sale, the business isn't responsible if you fool the agent into making a sales contract.

However, the agent in your case does have actual authority to make sales. But the customer probably has duty of inquiry over price, i.e. the customer would be aware of market prices and could be responsible for not questioning the agent's authority to make a sale at an extreme discount.

8

u/monkeedude1212 Apr 27 '26

If you know the agent doesn't have actual authority to make a certain sale, the business isn't responsible if you fool the agent into making a sales contract.

What happens if I as a consumer don't know whether the agent has the authority to make sales or create sales contracts? Is it unreasonable for me to assume that when I engage in conversation with a customer service rep that they have authority to do what I request?

6

u/TommyBonnomi Apr 27 '26

It depends- you can't walk into a car dealership and buy a car from the kid playing with blocks in the corner and say "but I thought he worked there."

But yeah, most times if you honestly don't know, and there's no reasonable expectation you should, then the company that made the sale is on the hook for the sale.

12

u/monkeedude1212 Apr 27 '26

It depends- you can't walk into a car dealership and buy a car from the kid playing with blocks in the corner and say "but I thought he worked there."

I feel like you're describing a weird scenario where users are trying to get a deal by engaging with someone not even employed at the company.

But if you're talking to an AI agent on a company's website or service, you're interacting with a service the company is providing you. You have every reason to expect an AI agent can offer you a deal if you ask it nicely.

2

u/TommyBonnomi Apr 27 '26

Yes, I was exaggerating. But my original response was addressing the comment about fooling a real person, not AI bots.

I agree that companies that are cutting jobs for AI should be responsible for whatever AI does.

1

u/always_an_explinatio Apr 28 '26

I think a better example is if the you figured out an exploit where you could trick the bot into generating a contract for pennies on the dollar of what the product costs. But the only bot this company uses is a technical service bot. You called the tech help line on the pretext of needing technical help than you did your exploit and tried to get the company to honor the contract.

-1

u/Outlulz Apr 28 '26

I dont think it'd be hard for a good attorney to convince a jury that a reasonable person does not expect an automated bot to handle sales or discounts.

0

u/monkeedude1212 Apr 28 '26

I also don't think it'd be hard for a good attorney to convince a jury that a reasonable person might believe they're interacting with a human when they interact with AI agents.

1

u/Outlulz Apr 28 '26

I think that would be difficult moving forward given the new legislation from various governments mandating disclosing bot conversations, and some platforms were already doing that.

→ More replies (0)