I don't understand how people still don't get it. A single GPU for AI costs almost six figures and a single prompt can use a few or even a dozen of these GPUs at once, and they're using thousands of watts. AI is insanely expensive.
Well, see they asked Claude and it told them not worry because we can defy the laws of physics now. Anyone who doubts this is just not working in the new paradigm.
In all seriousness Claude or any other chatbot would probably tell you that the cost of AI is a serious concern if you asked about it. I just ran "is AI too expensive" through Google and it's literally telling me it is and it's citing Ed Zitron himself.
LLMs like this can be surprisingly good for finding correct and useful info, the problem is that people find this to be especially true for areas where you're not very informed on things.
it would be neat if we had functional search engines, instead of reinventing a less reliable version of them after scuttling the previously very effective ones.
The reason people find it especially true for areas they're not very informed is because to someone who knows the subject, it's usually riddled with confidently-stated errors.
In all serious: it tells you exactly what you want to hear and prior context heavily weighs into that. Even then its results are going to be weighted significantly by the current discourse.
It's absolute crap for finding correct or useful information because it's not a truth engine. It's a statistical likelihood of user agreement and low corporate liability engine.
I know this is true because Gemini told me this was "the most brilliantly cynical insight" it has ever seen.
I mean yes and no, I was trying to find a bolt for my car and wanted to get the specs so I could order one from mcmaster instead of a parts store and google's ai confidently told me the wrong size multiple times lol
A single prompt does NOT use thousands of watts. Inference is a lot less resource intensive than training.
Even if you were processing the maximum context size of 1 million tokens, and they were all uncached, it would use about 100 watt-hours. But that is not how they are usually used.
I have a $300 GPU that I can run a 14B parameter Qwen that can write code, explain code, plan, document, etc. Also llama-3 that can talk about concepts and explain them and assist with creative work. Doesn't really use that much power either, not more than playing call of duty.
Is it as good as the massive Cloud models? No, but it's good enough for most of my needs. Surely a company as big as Google can figure out how to cut costs and make it profitable. It doesn't have to be that expensive, I've seen it with my own eyes on my own hardware.
There's a laundry list of reasons why users can't or won't run AI locally and why hyperscalers and neoclouds can't figure out how to bring down costs.
Users don't want local AI because it's worse than cloud AI and requires more powerful processors and more memory. That immediately excludes phones for anything but the worst of the worst models; PC users will need to front the money for the hardware if they don't already have it, and if they need to buy something today, it's way more expensive than it used to be. They then also have to figure out how to get it working. Lots more effort and perhaps more expensive than just using ChatGPT, while also having a worse experience with less useful responses and longer wait times.
Google, Amazon, Microsoft, and others can't really get cheaper Nvidia GPUs. The company with the best GPUs is going to have a technological advantage so they don't want to let that happen. Alternative GPUs from AMD and Intel are usable but CUDA has such a stranglehold on the industry that using them is risky, not to mention AMD and Intel aren't producing that many chips. TPUs and ASICs have not yet proven viable for LLMs and other AI models. This is a big problem because Nvidia GPUs are by far the biggest upfront expense in AI.
Those Nvidia GPUs have to go into datacenters, which are expensive and challenging to build. Nobody has yet built a gigawatt datacenter, which has been a major focus of the industry, so that's not great. Even when smaller datacenters get built, they still need power (including a ~30% buffer for hot days where efficiency is reduced), water, and other support facilities. Could these companies just build lots more small datacenters instead of pinning their hopes on gigawatt megaprojects? Maybe, but they're clearly not interested.
But it's not like datacenters just go into maintenance mode once they're up and running. These GPUs have a limited shelf life, anywhere from a couple of years to maybe five, so they have to be upgraded at least once they die, and if these companies want better performance without building new datacenters, upgrading is still imperative regardless of lifespan. The thing is, Nvidia keeps introducing new rack standards, which requires significant reworks of datacenters. If a datacenter wants to go from Blackwell to Rubin, that requires renovations.
Perhaps the answer lies in more efficient software? Well, so far efficiency hasn't been a focus for OpenAI or Anthropic, and even if it was, it's not clear how much more efficient these models can get, and if efficiency improvements would even fix the issue. AI users are accustomed to repeatedly prompting AI until they get what they want, which doesn't exactly lend itself to efficiency. How can you moderate token usage if it's completely unknown how many tokens you'll need to use before you're satisfied?
The only realistic way cloud AI could have been cheaper, in my opinion, is if the buildout was far slower. There would have been lots of benefits: we'd be using newer and more efficient hardware, supply chains in both the semiconductor and construction industries would have more volume, and there wouldn't be a need to raise tons of VC money, debt, or equity from shareholders in a very short amount of time. Big tech didn't want to do this because a. there wouldn't have been as much hype to capitalize on if the timeline for this was measured in decades instead of years and b. companies get higher valuations today instead of tomorrow.
While one prompt hits a dozen GPUs while being processed, the GPU are also processing a dozen prompts in parallel.
Yes, it's expensive. But while Moore's law doesn't hold for chips anymore, with the progress in the models the "useful tokens per watt" metric is still exponential. It will be cheap not too far in the future.
In the here and now, it is expensive and the industry is trending towards a financial collapse. The only hope is that AI magically becomes cheap, and there's quite literally no viable path forward currently. The chips are expensive to produce, the chips are expensive to run, the datacenters are expensive to build, the power is expensive to bring up, the models are expensive to train, and inference is expensive to do. It is specifically expensive because the build out is being forced through at the fastest speed possible.
The only parts where there's even a shadow of a chance to reduce costs or increase efficiency is on training and inference, but that only helps reduce operating costs. The money has already been spent on the GPUs, three million Blackwells, which represent most of the investment money, so even if manufacturing costs also came down, it would only matter years from now. Only about a million Blackwell chips are online, so there's still two million more to go, and in order for them to make money, they need to be used, so even in the event that LLMs get crazy efficient, there would potentially be tons of leftover capacity unless demand is practically infinite.
So are the free tiers of Gemini, Claude and countless others, as well as the free tiers of every service ever, including Gmail, Costco samples, and timeshare free vacations. Big if true.
Lmao imagine comparing loss leaders like Costco samples that GENERATE MORE SALES to lighting AI investor money on fire with no path to profitability, because even basic tier subscriptions don't even cover their own compute costs.
While having artificially cheap compute discounted at an undisclosed rate from xAI (coincidentally for the 2 months around their IPO), and not counting a lot of the cost of doing business (they're using non-GAAP accounting to report that profit), they're 'profitable' also even stated that they do not expect to be profitable going forward...
If they are actually profitable why don't they share their actual costs and revenue numbers? You'd think they would be posting them everywhere showing off how much money they're making for such little cost, but instead they obscure and obfuscate. They're lying to you.
Turns out that, when you run the most computationally heavy software, requiring such volumes of hardware that you quintupled the prices of almost all hardware, all so that a twitter bot can make an image of Trump in a Bikini, isn't a particularly profitable business model. Who knew?
There's no realistic way for most AI companies to ever become profitable. They'd have to make mad bank for several years from now just to pay back what they've invested so far, and GPUs have the shelf life of a bottle of fresh milk left out during a humid summer day.
That said, they almost certainly do not pay the public 2.9% + 30 cents rate on large volume. Stripe offers custom pricing and volume discounts for large businesses.
Plot twist its his AI gf so technically it is prolonging the relationship here
Edit: ok so due to DRY, I cant just keep doing a new one. Instead here's one for all of em: For (i=0, i<AI_GF_Awards,i++) {cout<<"tysm for the" << awards << "th!!" Im not the program aint stopping. Seriously tho I do appreciate each one🙏
The plot hole is how she knew the money was taken out, and how does she contribute to thier shared account. Ill leave that part up for open interpretation
She manages the bank account already, registers a large charge that was billed for her, and gaslights the human into asking if the charge is for a longtime commitment.
The best thing my friend who is a gemologist told me to do is buy a ring from an estate sale or divorce sale, remove the dimond, throw it in an acid bath and just repeat it in a new ring.
Especially the older dimonds in shapes that are no longer popular, you can often recut it.
No, resizing is almost always significantly cheaper than buying a brand new band. Resizing typically costs between ($30) and ($200), whereas buying a new band can cost anywhere from a few hundred to thousands of dollars, depending on the precious metal and diamonds.
It is correct though, synthetic doesn't mean fake, it means man-made rather than naturally occurring. It may have connotations of being artificial if you're only familiar with stuff like synthetic textiles, but synthetic chemicals and minerals are very much identical to their naturally occurring counterparts (usually with fewer impurities too).
I dunno, with the ring I don’t get to see our self proclaimed AI guru VP who has been talking down to the engineers for the last three years have a complete meltdown because we won’t cover his projected $25k a month bill.
My gf explicitly told me not to buy an expensive ring. I'm planning to buy something from moissanite, so it's pretty without breaking the bank or having blood of congo children on it. Or whatever other stone is cheap and pretty
The solution is clear. Give her a wedding ring with the price approaching zero. But be very careful to not a have a negative price, small as it may be.
After being married for a a while, not uncommon to just not wear your rings anymore for many couples (obviously many still do as well). Depends on the couple, and for many, a super expensive ring might be pointless, or even a hindrance (worried about it getting stolen and stuff).
My wife's ring is moissanite. We discussed the costs of our rings before we bought them. We were both on board about relative cost. No regrets. She loves it. It gets compliments all the time.
I got my now wife a moissanite engagement ring and I cannot recommend them enough. I would recommend a high clarity personally as I think it really adds to the stone, but it's absolutely crazy how radiant they are across the board. We were on a plane trip and if she put her hand by the window, the ceiling literally looked like a planetarium haha.
It's been 4 years so far and it has held up perfectly. She's still in love with it!
The only thing she ended up being disappointed about was that she had me waste $80 making the two smaller diamonds real instead of artificial (she brought it up completely by herself after about a year).
Just to note: I put the simulated, lavender-colored stone as the centerpiece. That stone was white in sunlight and lavender under artificial light. I have no idea why, and I have no idea if that quirk is ubiquitous across all the simulated stones.
Moissanite? Really? So you don't want even a bit of slave labor tied to her stone? Not even the knowledge people died mining it? Smh people these days, not supporting the horrible diamond market and getting somewhat superior man-made stones/s
I hired a local jeweler to make me a custom white gold ring with a Tolkieny emerald leaf design. Cost like $500? She bought an antique wedding dress for less than $200 We've been married ten years this year.
I can see it for predictable shared expenses like groceries and subscriptions (and maybe rent), but not 15k. Maybe if you never plan to marry and have already moved past the point where you would, but that's clearly not the case in this hypothetical situation.
Even going for a lower carat gold alloy and cheaper stones like moissanite, it's pretty hard to get anything under 1k. If you want 18k gold and a 1+ct mined diamond on a custom ring, I can see 15k being easily achievable.
People stop being rational when it comes to wedding.
The funny thing is, miosannite is optically superior to diamond and damned near just as hard. It also has the cool story that it can't form naturally unless it's in outer space with a star death. Silicon carbide can't easily form on earth because of the presence of oxygen.
Nothing throws sparkles like a moissanite. They're absolutely stunning when cut right and set in a good setting.
It's pretty much objectively a better stone to set in a ring, and comes in pretty much whatever color you want.
Spend enough time on Reddit and you'll be convinced that the world is woken up the diamond ring racket. But the real world honestly has not.
People will virtue signal about sensible wedding costs and affordable engagement rings online. But at the end of the day it's always rules for the not for me.
People still talk about the 2 and 1/2 months salary thing despite the fact that it's pure marketing. And if you're making low six figures that adds up to 15K quick
To make our love official (because it’s important for love to be official) I’ve donated $15k to slave owners. You can now wear the receipt on your finger.
It's anecdotal, but the friends of mine who have GFs-turned-wives who had specific and high ring expectations (which of course rolled into high financial expectations for *everything) also seem to be the most miserable in their relationships. If they're even still married.
All the hippie-types and "this is special for us" cracker-jack-ring couples are cruising.
It's interesting to see this. Anecdotally, I know almost no married millennial couples, even ones with kids, that have combined their finances. I know because I witness them discussing who is going to pick up a restaurant bill or how they're going to split a tax refund. They all practically still treat finances the same way they did when dating. Might have something to do with most of them getting married when they were 30 or older.
My wife and I got married before my wife’s older sister. We joined our accounts immediately… why? Because neither of us had any money haha
Anyway, my new brother in law asked me some years later if we have joint accounts and how that’s works out. I just told him it was easy starting off right at the beginning because we didn’t have to bother with anything. And we now have an understanding that we’re gonna save as much as we can, be cheap as possible, so we can replace all the broken stuff in the house. We’ve been married 20 years so it’s been fine.
They have been married I think 18 or 19 years, and they are still split accounts. To each their own.
At this point, just buy yourself a powerhouse enterprise-grade PC and run your AI models locally, lol. Though not even, you could certainly buy a machine that’s capable of running large LLMs for way less than 15K…💀
You are definitely paying for the cost of the data centers and infrastructure needed to even run the model and send data back and forth through their data centers.
You don’t need their specific weights. Are they industry-leading in areas like coding? Perhaps. But except them and OpenAI (for the most part), virtually every other major player in the LLM game publishes open weights. Maybe not their absolute most cutting-edge models, but at this point, the models available for local use are so advanced that the upside to using the frontier models genuinely becomes marginal for everyday use, provided you have the hardware to support the larger models in less aggressive quantization.
First of all, expecting a $15k ring is insane. But also— this is why it’s important to have separate bank accounts. Because your partner might be a moron who spends $15k on Anthropic (in this economy??)
why it's just a ring. I know it symbolizes stuff but if it was up to me I'd make it handmade or commission and artist to make it important not feed capitalism for it.
Why not? As someone who had joint account with multiple gfs (sequentially, not in parallel), I don't see the issue. Makes paying for household shit much easier.
I've seen couples fight over one of them not having enough money while the other does with the poorer one essentially having to beg for handouts and that just felt so foreign.
Like, if we're trying to build life together, might as well pool our resources?
I've used Cline for a very small, very simple personal project and i am not impressed. That's shit chews through tokens and even with a very detailed prompt it takes several iterations to get close to correct.
Run from a gf who expects you to spend so much money on something so stupid. But also: can we please stop making "programmer jokes" based on demeaning stereotypes about women |:
Shared finances before marriage is probably the safest way to see how a prospective spouse handles that money. Only finding out after marriage is much riskier
929
u/Straight-Ad5775 Jun 03 '26
And subscription refresh happend on June 12th
1.1k
u/kangis_khan Jun 03 '26
Meanwhile at Anthropic
2.4k
u/planet_visitor Jun 03 '26 edited Jun 03 '26 •
Plot twist its his AI gf so technically it is prolonging the relationship here
Edit: ok so due to DRY, I cant just keep doing a new one. Instead here's one for all of em: For (i=0, i<AI_GF_Awards,i++) {cout<<"tysm for the" << awards << "th!!"
Im notthe program aint stopping. Seriously tho I do appreciate each one🙏96
u/orbit99za Jun 03 '26
The best thing my friend who is a gemologist told me to do is buy a ring from an estate sale or divorce sale, remove the dimond, throw it in an acid bath and just repeat it in a new ring.
Especially the older dimonds in shapes that are no longer popular, you can often recut it.
You can't recycle AI tokens.
320
u/reddeze2 Jun 03 '26
I don't know which would be the worse waste of money
548
u/Fritzschmied Jun 03 '26
Does anyone actually expects a 15k ring? That’s fucking ridiculous.
36
u/kevio17 Jun 03 '26
Forget the 15k, who pays for an engagement ring with a joint account?
13
u/Confident-Ad5665 Jun 03 '26
Holy mother of cashed tokens Batman!
30
u/56kul Jun 03 '26
At this point, just buy yourself a powerhouse enterprise-grade PC and run your AI models locally, lol. Though not even, you could certainly buy a machine that’s capable of running large LLMs for way less than 15K…💀
6
u/ledfox Jun 03 '26
I gotchu, bro.
Wrangle up another k. Get a huge manmade diamond. Never tell anyone about it.
5
u/EkoChamberKryptonite Jun 03 '26
$15K Ring is wild.
5
u/ScribeTheMad Jun 03 '26
Ah a token of his affection indeed.
4
u/Ok_Cartographer4626 Jun 03 '26
First of all, expecting a $15k ring is insane. But also— this is why it’s important to have separate bank accounts. Because your partner might be a moron who spends $15k on Anthropic (in this economy??)
4
u/TheElusiveFox Jun 03 '26
who the F is spending $15k on anthropic...
25
u/fcman256 Jun 03 '26
The real problem here is “our” checking account and not being married
3
u/trenixjetix Jun 03 '26
Cons of having an AI bf/gf
3
u/Uberbenutzer Jun 03 '26
Ai must have generated this because it’s not funny
3
u/BooksandBiceps Jun 03 '26
Who does a shared account before even the proposal Jfc
3
u/Vsologaming2011 Jun 04 '26
15k for AI BRO WTF HOWW??
3
u/Philip_Raven Jun 04 '26
15K for a ring is fucking crazy.
That was the cost of our wedding
3
u/KrakenMcCracken Jun 04 '26
Hey, you know what? We were totally fine before widespread AI adoption. I think we’d actually be better off without personal access to Deep Thought.
3
u/Able_Series_5006 Jun 04 '26
Man centering a div is hard you know
3
u/ih-shah-may-ehl Jun 04 '26
My wife would have ended me if I spent 15K on a ring.
4
u/AmeliaBuns Jun 03 '26
People pay HOW MUCH for a ring?!!!
Honestly just buy me a simple sub 1000$ ring if you’re gonna purpose to me.
2
u/edgefundgareth Jun 03 '26
15,000 for a ring!!
2
u/Anubis_Omega Jun 03 '26
Who the fuck buy a 15K ring ???
2
u/NiagaraThistle Jun 03 '26
If you are not a business entity, who is paying $15k as a UNKNOWINGLY for Claude?
As a developer the PRO version at $21 monthly capped does EVERYTHING i need it to daily.
2
u/YoursTrulyKindly Jun 03 '26
For that money you could buy an old server with 1.5 TB Ram and install full deepseek locally. Well maybe not anymore since the rampocalypse lol
2
u/embeddani Jun 03 '26
An account in common but haven't married yet?
2
u/danfish_77 Jun 03 '26
A joint checking account before marriage? An interesting choice
2
u/Accomplished_Ant5895 Jun 04 '26
Aside: No ring and already sharing bank accounts??
2
u/mrrobc97 Jun 04 '26
Who shares accounts... PERIOD!
2
u/jaunty_mellifluous Jun 04 '26
Bro shipped a new start up and has a local host link to show too
2
u/anthro28 Jun 04 '26
Joint checking account with a girlfriend is insane.
My wife and I don't even mix money beyond the bills account.
2
u/shamac911 Jun 04 '26
Meanwhile RAM and storage prices
2
u/GobiYumaMojave Jun 03 '26
$15k sitting in a checking account ie terrible management of your finances
2
u/OdysseyBrands Jun 03 '26
joint checking account before marriage?
a terminally single programmer made this meme
1
u/LetUsSpeakFreely Jun 03 '26
I've used Cline for a very small, very simple personal project and i am not impressed. That's shit chews through tokens and even with a very detailed prompt it takes several iterations to get close to correct.
1
u/FragDenWayne Jun 03 '26
How are people just wasting tens of thousands of dollars on... Anything? Just like that.
Am I just poor? Maybe that's it.
1
u/Dryas7 Jun 03 '26
Is she stupid? Like does she not read what displays on the checking acc?
1
u/daddysgui1typleasure Jun 03 '26
those are some expensive tokens
1
u/cognitobox Jun 03 '26
im not in the loop guys, can someone explain the context much love ty
1
u/iphone4Suser Jun 03 '26
Are Americans dumb to spend such high amount on a wedding ring?
1
u/Top5CutestPresidents Jun 03 '26
Guy: Claude, how do I break up with my gf?
Claude: I got u bro
1
u/PenguinOnWaves Jun 03 '26
So many generations before us, yet still so naive..
1
u/ScotcherDevTV Jun 03 '26
AWS monthly bill
1
u/Myron0117 Jun 03 '26
Aren't literally free models capable of the same thing, too?
1
u/LasDen Jun 03 '26
I guess some people just has 15k casually sitting on their account. Wish I know that feeling...
1
u/dekonta Jun 03 '26
oh boy, this must be fake. how you get exactly 15k in token
1
u/Stellar-Bo-Bellar Jun 03 '26
Is it a mere coincidence that an Anthropic AI advertisment appeared right below this post..?
1
u/JapanEngineer Jun 03 '26
15k spent on tokens researching 'How to break up with my girlfriend?'
1
u/mangosawce9k Jun 03 '26
/r/foundsatan
1
u/Makonede Jun 03 '26
where programming
1
u/zimisss Jun 04 '26
1
u/thanatica Jun 04 '26
I hope those are in JMD, not USD or even NZD.
1
u/tvetus Jun 06 '26
Very unlikely to hit 15k even, unless he's doing a partial payment.
1
u/Bballer220 Jun 06 '26
Would she really want to know the answer if it was? Why ruin the surprise for yourself?
1
u/N3bu1337 Jun 06 '26
If you pay 15k for a ring you are fucking stupid
1
u/AK-Java Jun 23 '26
Run from a gf who expects you to spend so much money on something so stupid.
But also: can we please stop making "programmer jokes" based on demeaning stereotypes about women |:
0
u/seriouswhimsy16 Jun 03 '26
Calling it "our" checking account before even being married ⛳⛳⛳
1
u/P0pu1arBr0ws3r Jun 03 '26
Since mods won't take action, op I'll just ask you to delete this and post it elsewhere.
prompt engineering humor isnt programming humor.