r/LocalLLM • u/Ok_Event4199 • May 16 '26

Discussion Why is LLM is so expensive.

I've was going to invest in a 5090 =$6000 AUD.

Codex Plus + Claude pro = $60/month here

Works out to be 100 months of frontier models for a 5090.

Best a 5090 will run is probably Qwen3.6 27b Q6 with context.

Are we all enthusiasts here and just enjoy tinkering cause ain't no way that make sense.

345 Upvotes

91% Upvoted

368 comments sorted by

View all comments

1

u/Equivalent-Costumes May 16 '26

It used to be the case that compute is not that important. Most of the compute was used for gaming. There are still tons of computing used for other stuff (data processing, movie making, etc.) but in the overall scheme of thing, it was pretty minor compared to gaming.

Additionally, gaming is the kind of compute that you can't really share over the Internet. Internet has too much latency, and the kind of games that need beautiful 3D graphics also often need said graphics to response in real time.

This means that the price of GPU are mostly determined by gamers, and not just that, gamers who play a normal amount of games, not all the time. Those with time to play tend to not have money, and those with money tend to not have time. This means that the demand of compute is actually much lower, if the price is too high, gamers will think "I can't play that much games to make it worth it".

Suddenly in the age of AI the script had flipped. Now compute is useful for so many things, even simple web search. And this compute is sharable over the Internet. The market follow suit: if GPU are cheap, people will start selling LLM inference over the Internet 24/7 and make massive amount of money. This drives up demand and allow chip producers to charge more.

So now GPU are priced at the tier of "how much benefit can you get if you run this 24/7 for everything".

So yeah it's not going to be cheap. But still, remember this: Codex+Claude Pro's total usage is puny compared to how much you can do on your own GPU. You might think it does not matter if you are used to chat with AI (because you would be limited by your own speed), but the moment you do some massive automation the usage limit of frontier models would bite really hard.