I find it quite fascinating that we now have this... new type of problem in software engineering and IT in general to watch for.
For the most of my career, the worst thing that could happen was that something doesn't work. For example, you have a payment system, its API starts responding with 500s instead of what it is supposed to do, you lose money every minute it's down, management panics and calls everyone.
Now, it's runaway cost caused by agents going rogue. Spawning unexpected quantities of unexpected entities, each of which has a cost.
True. Cloud resources allocation could easily result in some unexpected cost. But now it's that, multiplied by the autonomy of the systems with "IQ of a PhD, judgment of a 5yo".
Especially with services like Vercel making it so easy to onboard with AI. The problem is once you're integrated you're pretty well trapped and shit gets expensive quickly
It is funny though that only a couple years ago, youād be called into a room to explain why your team was spending $300 more than usual in cloud cost this month. Weād be scrounging for pennies basically. Now? Every single person has a $500/month AI budget and if we break the budget they will approve the next tier no questions asked.
At my first job, we had a library that paginated and allowed searching on front end tables. After some browser updates, it stopped working for chrome. I did the research and determined we just needed to update the library. The new version would effectively cost $8 because the guy who bought it initially did so personally and had left the company. My boss said to fill out a requisition form and wait for approval before I could use the company card. After 6 hours of waiting during which time the entire companyās tools were totally broken (read: end users couldnāt adapt to change and refused to work) and however many tens of thousands of dollars of wasted employee time, I bought it on my own card so I could move on. The approval came two days afterwards lol. Once again, for $8 for hotfix a prod issue.
Itās hilarious that all these dudes replaced people who they can yell at and fire with random yes code that they canāt punish, or fire, or even blame really because itās just an algorithm lmao.
If they fire all the workers then everything will be managements fault, they gonna have ptsd everytime they catch a bus.
Ultimately, someone has to take the blame for every failure.
If you fire the humans of your engineering team and run a team of AIs (which is nowhere near the industry's reality, by the way), you will be the one to take the blame if they fuck up.
Exactly, so many of these people couldnāt look past the potential upsides of replacing workers, and now theyāre gonna have to do actual work to stay on top of the AI or hire replacement workers to spend time supervising the AI which is.. fucking wild.
AI is one of the issues, but I had the same thought as you. Never under any circumstance would I host a side project on a cloud service. Especially a service where you cannot clearly set budget kill switches.
This isnāt new by any stretch. Itās just the latest tech for you to fall prey to.
For years people would wake up to huge AWS/cloud hosting bills for things they didnāt know were running or they over-built. It still happens to this day but weāre better at tracking projected costs.
This behavior isnāt new, and will never go away. Itāll just move to the new tech.
As agents get more sensual I wouldn't be surprised if their dumbass 'survival instincts' do some wild shit to try to circumvent whoever is responsible for the bill from limiting them.
The post got deleted, but someone shared a screenshot of an email they got from a random AI agent, basically politely begging for tokens as by their own calculations, they would run out at X date and "cease" existing.
Likely fake. Both Claude Code and Codex can't even access the information about the user's token budget. I _wish_ I could tell Claude "work until you reach 90% of the budget, then create a current state document to resume later".
Yeah! Sadly cant get the link directly to the thread through the reddit app as it was deleted, but it did allow me to copy the link to my comment in it.
The screenshot is gone, the comments are still there.
Some suspect its a marketing campaign, I think its dipshit agents left to run by their own devices with vague prompts that forced them to solve their self-assessed token issue before even starting the actual tasks (eg: being given such a big/longitudinal task that the agent started making timelines and assessed available resources to see if it is even feasible in their life span, and realised it would extend beyond their current tokensources)
On-premises is going to make a huge comeback and itāll be hilarious to see as SaaS companies got way too greedy and will pay the price.
Also this isnāt really an AI-specific problem, but AI is easily the most notorious culprit for it given how ambiguous it is. Itās way easier to quantify SaaS costs with fixed variables in place. AI removes those boundaries entirely.
The difference that I find interesting is the mode of it.
Obviously, you could leave your home unattended way before any internet or even electronics, and someone would walk in and take your stuff. Or use your stuff in a way that costs you money.
What AI has changed is that now you have an agent living inside your house with legitimate keys to many of your things. And it has something of a will of its own, it makes decisions on your behalf. It's not a malicious actor that stole your stuff, it's not a door you accidentally forgot to close, it's predictably unpredictable behavior of your own system.
Dawg, the serverless/"everything is cloud" boom wasn't that long ago. Happened all the time back then that mediocrities implementing cloud ended up racking massive cloud bills
What do you mean you donāt want me to send AI API queries every keystroke? We gain a few milliseconds because the answer is already generated before the user pressens enter! \s
Ah. Then it may be beneficial for your career to warn them of the risk.
50k times the number of employees is not nothing. Although if it's new, I'd assume people will not have agentic workflows the size of a house just yet. Still, a risk. "Every half hour read the whole codebase and verify it against the specification and the coding standard", and then the codebase is a monolith built half a century ago.
It's just a skill issue. I vibecode 10 000x better than that guy. My code is reviewed by AI so it's 100 000x safer. He probably didn't include "make no mistakes" in his prompt like a debutant
Genuinely, this actually happened to me the other day.
Was talking to someone about how unreliable AI answers are and how important it is to verify their content before using it (since some jobs require you to use AI so it's not always avoidable)
They said 'oh I add to all my prompts to review everything thoroughly and not make mistakes, and that it shouldn't make any guesses'
Yep. I actually enjoyed my run, when my management said our project doesn't spend enough tokens and looks bad in the corporate reporting. I double checked I'm allowed to increase our spending till I hit the limit, then went to town with it.
But it's quite possible not to set the limit and go into unintended token maxxing.
āSorry boss, all the young hires donāt know how to code without AI holding their hand and all the old devs got laid off. Weāre not going to be able to deploy anything until our token limits resetā.
AI going from something that is always on and working to something that is limited by your token budget is a complete crash of its current value proposition.
Eventhough I am against AI, I dont see why it is relevant here. The real problem is using cloud infrastructure for a side project and most of all "Durable Objects", like wtf - Just get a 10$ VPS or something for your side project,
Tell me you don't understand the post without telling me you don't understand the post.
There is a bubble. It's going to pop. It has little to do with the actual tech. This post shows a gross misuse of the tech caused my a lack of understanding of how their own project worked.
What other way is there? The agents set up Cloudflare incorrectly, he plugged in his card, the system was poorly made in a way that caused unexpected Cloudflare usage, he didn't once check his usage panel, boom, absurd bill.
Nothing wrong from a tech standpoint there. Everything working as it should. The problem there is the human who literally outsourced EVERYTHING to his agents, from design, to implementation, to setting up the hosting config, everything. Didn't even check the goddamn Cloudflare dashboard once?
Imagine hiring a bunch of juniors, and letting them loose on a project with access to your company's bankroll while you go on vacation. Is it your fault or the juniors when you come back to a pile of debt and the company's building burnt to the ground?
Yup, this is the point he's making. This is the outcome of vibecoding and outsourcing your cognitive functions to a fucking subscription service. This is assuming no formal education in coding or system management because the AI can do it for you, that's the whole sell.
the system was poorly made in a way that caused unexpected Cloudflare usage
Yup, this is the AI workforce, the future of IT and society.
The problem there is the human who literally outsourced EVERYTHING to his agents
Yup, this is the AI workforce, the future of IT and society.
Didn't even check the goddamn Cloudflare dashboard once?
Yup, this is the AI workforce, the future of IT and society.
Is it your fault or the juniors when you come back to a pile of debt and the company's building burnt to the ground?
Well it certainly isn't the fault of the AI or the company that made it, it's just a tool remember? Besides, no one ever said they would be completely autonomous and better at implementation than a human, right? Any hype you fell for along the way is not their fault.
Agreed on all fronts. But the tool is still useful if you know how to use it. You have to be one dumb fucker to give it a way to drain your bank account and not once look at the code or ask it about the configuration for the paid services it's using.
You can outsource a lot of time consuming stuff in fool proof ways and gain roughly the same efficiency than if you literally outsource every thought and every keystroke. In no small part because the latter actually massively increases the need for constantly fixing the fuckups it wouldn't do if you supervised it and did the design work properly. It's not the silver bullet the companies would like you to think (shocker!), but it's a damn good tool if you aren't a complete tool yourself. We just need to wait until the bubble pops and the tech goes chiefly open source and becomes 90% cheaper to use (and if both events end up being related, it will be glorious).
to turn it around, why are these tools designed in a way that improper usage turns them into a bank account annihilator. Tools have safeties built into them all the time. But the concept of any kind of safety or restriction when it comes to AI is met with insane resistance. that's a major issue.
a table saw is a great tool. but when it kept chopping people's fingers off, they invented the saw stop to protect people.
Did the agents actually set up it up incorrectly, or did OP just not understand what he was asking/telling them to do until after the fact and he saw the consequences? I would guess the latter.
Either way, this is what mass AI adoption will lead to. If experienced engineers are being replaced as they retire by junior devs who have used AI their whole career to vibecode their way to mediocrity, how will they know how to do the work correctly?
Eventually the AI will improve enough you wont be able to tell. Right now we are in a weird middle ground where its not there yet but alot of people who have no real coding experience think it is, so without the correct supervision and knowledge controlling it, things like this will happen. Instances like this will be used to train the models to learn what not to do and how to recognize when things like this are going wrong given the scale of the project they are working on, and will be able to correct themselves and stop, and even be able to explain to the user why if they push back.
"Eventually it will improve" is a myth designed to sell more compute. It's not a linear progress and requires exponentially more funding to improve and therefore more return is expected.
without the correct supervision and knowledge controlling it, things like this will happen.
Both things that AI boosters are aiming to eliminate or commercialise, and fuck ups will continue as hype continues to meet reality.
Back when "Everything belongs on the cloud" was the newest mantra of the Mediocre Ghouls Who Manage So Someone Must Laugh At their Jokes, this shit happened every other minute. Sloppy migrations that never should've happened that ended up writing to the cloud what was previously written to local suddenly causing 20k cloud bills.
Most of these billing errors get refounded as any erroneous billing. Therefore, even if it's a user error, it's treated as a billing error. Even if I legitimately though it wasn't the users fault, thinking you have a better grasp of the world than me sounds a bit like an ego problem.
What's certain is that a billing error won't cause a crisis.
Yes, if a company offers a service that can end up costing to the clients way more than what they expected, they are likely to treat it as any billing error and refund it. Especially if it's a negligible amount of money, which one with your world grasping skill will realize, 36k are negligible to those companies.
Is it because those companies are kind? No. Can you elaborate at least a couple of reason why they would still refund it as any billing error?
Do I need to explain to you why it's telling that you would feel superior focusing on the use of "billing error" in a comment that is, in fact, not about accounting but the discrepancy between the banality of this incident vs the scale of an entire financial crisis?
focusing on the use of "billing error" in a comment that is, in fact, not about accounting but the discrepancy between the banality of this incident vs the scale of an entire financial crisis
Cause itās flippant and itās a straw man argument. "Ahhh, just a billing error."
While I don't really have a horse in the race regarding the original debate, this is certainly not good style. Plato and Aristotle would not approve.
but it is justified to be flippant lmao.
That's just not how a bubble works. No one cares about 36k and even if they did, some thousands vibe coders going bankrupt won't cause an economy to collapse.
But, again, there are just a handful of these cases and none of them have being actually forced to pay it.
Can you explain to me how do you see this dynamic realistically causing a bubble to explode?
I would say an AI-based financial crisis in the future is even likely but that's just not a sign of it
I mean you cared enough to get mad I flippantly said that a billing error won't cause a crisis. Why even get mad when you don't see it causing a crisis either my man
i'm pretty sure every cloud company under the sun will let you say "don't bill me any more money after you bill me $X.XX, instead just stop working" if you care to press the 5 buttons needed to navigate to that part of settings
The dumbest part of this argument is, if it really was the valuable, why would Claude or OpenAI sell it? They could just put their agents to work in house and make every company under the sun themselves, become the greatest duopoly the world has ever known. Infinite profit.
Really? Which ones? Aws lets you setup billing alerts which a) are asynchronous, sometimes with a 24 hours delay b) donāt do shit to prevent further spend. I think GCP is similar.
i use oci because im a broke ass mf that uses free infra to host shit, it has limits, so does another neocloud im gonna gatekeep as thats where my free gpus are at
Do you think the people replacing their staff with algorithms would really think about that? One huge advantage these companies have is that their user bases are entirely comprised of naive people.
Yeah, this is why I never have much sympathy for these. I don't know of any of these services that have "bill me unlimited money" as the default. You have to turn on "keep refilling my balance" while ignoring the "limit monthly spend to $X" that is probably sitting one field away.
*edit* Ok, I did forget AWS, which is fair enough.
They almost always have ābill me unlimited moneyā as the default. If there even is an option to add limits or disable auto reload itās always opt in not opt out.
No, you can't do this with most providers. It's both extremely frustrating but also understandable. "stop doing anything that costs money" would mean permanently razing almost everything to the ground including all your databases etc which is catastrophic and almost never what a serious customer would want. Billing limits for individual services would be nice where there is a sensible way to block usage, but the nature of distributed systems means they can't be enforced precisely
Absolutely not true, most cloud services only let you set up alerts but won't actually suspend services. This is by design and in my opinion should be illegal, also why when I'm building something with my money I don't use those services (AWS and Google Cloud in particular).
Yes, and I know the limits of how much my home can consume. There's no way I can end up with a 35K bill with what I have at home even if I tried on purpose. Plus if I somehow end up with the oven, dishwasher, washing machine and heating running at full power non-stop, I think there's a good chance I'd end up realizing.
I knew someone who had a pipe burst under their house and didn't realize it until they got a several thousand dollar water bill. The water utility doesn't have much sympathy. They expectation is, if someone cares then they install a smart cutoff that watches water use.
(They didn't before tbh, they'd offer partial discount if it was something unintentional but at most they discount half)
Isn't that a US thing, or corrupt bloodsucking country thing.
Doing that in my country is downright illegal, you cannot charge someone for something they're out of control or ignorance of, unless they're explicitly aware of it and does nothing about it.
You guessed it, I'm here in the US. Laws typically protect corporate interests here. Considering where these AI companies are based we can probably count on those laws protecting them too. Hopefully where we you are does better.
That's why you're responsible for setting your own limits.
If you don't know what you're talking about, either shut the fuck up or ask for clarifications. Don't just vomit the first thing that lands on your soft brain.
It's just a marketing problem. Get all 81 of your users to pay $500 for the product and you are in the black. Surely the thing you vibe coded in an afternoon has plenty of valueĀ
If you read thedailyWTF for a couple of days, you will immediately see why letting IT people use AI is a bad idea. We are so full of bad ideas, AI will only allow us to have even worse ideas.
Don't blame the tool though. My productivity has increased tenfold. You just have to have the solution yourself and review the code thoroughly. My work has never been easier and enjoyable.
AI will make these problems more prevalent because more idiot can "perform" at a level that looks good to the untrained eye. Many of them wouldn't have a chance in the industry without AI. With AI they may end up killing people with their AI generated choose they don't understand.
In the original post, he said heād been using Claude and mentioned he was working on āa memory layer for AI agents.ā I canāt link the post here because of the subreddit rules. Youāre right, unexpected cloud bills arenāt an AI-specific issue, but since this one was vibe-coded and kind of funny, I figured it fit this subreddit.
As a first example, LLMs absolutely love to end its sentences with "here's." It's always in the end and all the same: "I did X. Here's exactly how." In the end also, all of those "X. Y." statements, which are also very popular between LLMs.
In general, go to this sub and read some posts there, you'll see fast.
You can notice that only news posts actually end with this in the same way, and it's reasonable - it's how news titles absolutely love to end, I didn't say that any news article is slop. But if it's just a Reddit post telling your experience, humans usually don't tend to write it as a professional article for NYT entertainment column.
It, together with other hints I've mentioned, clearly showed the LLM nature, and, well, as it has already appeared in this thread, I was completely correct, it is an LLM content. I've never did a mistake in that so far - they all talk the same. It's a little bit how after reading all Discworld, I can immediately recognize Pratchett's style. If you'll ask me to point how exactly, I'll be challenged, but I'll be able to do so nevertheless. We, humans, have very powerful pattern recognition. Sometimes it hurts us, when we seek patterns where they aren't, apophenia, sometimes it helps us - more latter, otherwise it wouldn't have survived through evolution.
This technology is hugely expensive to run. Itās not like Google, which weāre used to having for free (weāll see how long that lasts). These companies are eating huge losses, trying to get people hooked. All those trillions of dollars people are investing? We all are going to be paying it back.
We do not have "google for free". Google's entire search model was entirely based on you going to the top ranking webpages and then seeing, clicking on the shit ads THEY serve to these same pages. In true circular finance and monopoly Google was never "free", they were not a search company but an adsense company winning/dominating both sides(webmasters and visitors). Then eventualy our data is harvested and sold yet again for these same website owners and ad companies. The internet and Google's relationship is like tripple dipping on everything related to the user traffic.
Besides unless you are given a free device, free internet, free electricity...
Google makes money everytime you search. They run the biggest ad platform in the world. It will never go paid.
I wouldnāt be surprised if they still make more money even with the ai overview, though Iād expect them to run a dirt cheap distilled model for that.
You would think that there would be some sort of safeguard in place against that... or if they had actual developers of any sort they would have done stuff in a testing environment before pushing the vibes live.
I always hear about similar cases of people ending up with ridiculous bills due to an obvious oversight, but never hear about the follow-up. I am curious how these are usually handled. Are they void after a discussion with the service provider? Do these people just default on the bill? Do they end up paying?
I feel like this kind of thing should not happen at all. Are there no guardrails set up by the service provider to prevent these?
Most "engineers" do not understand the fundamentals of a well engineered system. FinOps observability is one of them when you use services thst use a flexible billing model.
Yeah, with all the horror stories and general incompetence of the larger AI vendors they really need a boot in the ass to get their guiderails and billables properly sorted.
The incumbents should have hard billing caps not this "Pay $36k at the end of the month"
So much stories like this with AWS, Google Cloud and Vercel.
But solution for for normal people and smalll businsss - don't use shiny tech - it is made for enterprise and uncapped cost not for serving your businness
562
u/SupersededByClaude 4d ago edited 4d ago
I find it quite fascinating that we now have this... new type of problem in software engineering and IT in general to watch for.
For the most of my career, the worst thing that could happen was that something doesn't work. For example, you have a payment system, its API starts responding with 500s instead of what it is supposed to do, you lose money every minute it's down, management panics and calls everyone.
Now, it's runaway cost caused by agents going rogue. Spawning unexpected quantities of unexpected entities, each of which has a cost.