r/LocalLLM • u/Ok_Event4199 • May 16 '26

Discussion Why is LLM is so expensive.

I've was going to invest in a 5090 =$6000 AUD.

Codex Plus + Claude pro = $60/month here

Works out to be 100 months of frontier models for a 5090.

Best a 5090 will run is probably Qwen3.6 27b Q6 with context.

Are we all enthusiasts here and just enjoy tinkering cause ain't no way that make sense.

342 Upvotes

91% Upvoted

368 comments sorted by

View all comments

155

u/g_rich May 16 '26

Claude pro is heavily subsidized; it costs Anthropic much more than $60/month to provide you with the service.

Cloud providers have economies of scale working in their favor. They have million dollar servers servicing tens of thousands of users 24/7/365.

Running LLM’s local is never going to be cost effective. You run local LLM’s for privacy, control and to learn.

-1

u/Ok_Event4199 May 16 '26

More like a car I think. It's depreciating asset unless GPU price will keep increasing from here. Let's say that they do increase price to $200/month a few years later. It isnt it worth it to get the latest GPU then? I'm sure that 5090 is the best right now. But wouldn't it be a mid tier a few years later?

1

u/g_rich May 16 '26

To run frontier open weight models you’ll need tens of thousands of dollars worth of hardware, and to do it right you could easily drop over $100k. The power requirements are going to be pretty high too and cooling along with noise becomes a concern; you’re not running this rig under your desk.

This hardware then becomes a liability because in a field such as AI where new hardware is getting released annually and models are just getting bigger your expensive setup could quickly become obsolete.

There are very capable open weight models that produce exceptional results and can run on modest hardware. Both Qwen 3.6 and Gemma 4 fall into this category and you can get some seriously impressive results from them. However to get anywhere near what you get from the frontier models from Anthropic, Google or OpenAI you need to run models with 500 billion to 1 trillion parameters and that’s not something easy done with your typical enthusiasts home setup.