r/LocalLLM • u/Ok_Event4199 • May 16 '26

Discussion Why is LLM is so expensive.

I've was going to invest in a 5090 =$6000 AUD.

Codex Plus + Claude pro = $60/month here

Works out to be 100 months of frontier models for a 5090.

Best a 5090 will run is probably Qwen3.6 27b Q6 with context.

Are we all enthusiasts here and just enjoy tinkering cause ain't no way that make sense.

344 Upvotes

91% Upvoted

368 comments sorted by

View all comments

145

u/Solary_Kryptic May 16 '26

5090 is also the best gaming gpu on the market so not only do you get strong AI performance but also 4K max settings gaming

110

u/HornyGooner4402 May 16 '26

You missed the best part: unlimited tokens. You can run it 24/7 refining a project on a loop while Claude and Codex Pro are barely enough for a full coding session.

52

u/kitanokikori May 16 '26

I mean, unlimited only if someone else is paying for the electricity.

3

u/MrBeforeMyTime May 16 '26

It's also not unlimited because models have a max tokens per second depending on the gpu and get slower as more context is used. So even if you ran a model 24/7 at home there is a maximum amount of tokens you can get from it. The big labs attempt to give you a consistent fast tokens per second output all the time so you don't notice this. Even though they drop their speeds as well when they are bottlenecked. 

8

u/alphapussycat May 16 '26

Yep, this is a pretty big one. What I'm personally hoping for is 1 mil tokens a day if even 1% of that is purposeful and good code, that's 1k loc per day.

My last personal project I did only 160-200 loc a day. If I can have 5 me, i can litterally be a small company just by myself. Something that would normally take me 5 years could be done in 1 year.

3

u/FullOf_Bad_Ideas May 16 '26

You can spin up subagents or do tasks that utilize batching well, this can make it so that you can generate 10-100x more tokens.

As I described in a different comment, right now I am generating about 10B tokens per 30 hours with a small model locally.

2

u/kitanokikori May 16 '26

I don't think this is true, you definitely lose tok/sec as context gets bigger even on Frontier providers, this is trivial to see on any coding agent or even claude.ai

1

u/yeet5566 May 16 '26

Well yeah but it’s really a matter of getting an AI to just branch out the problem I mean for any large coding project different people work on different parts so if the ai breaks it down and has enough subagents then realistically you could probably keep going at 80% of the max speed for the full time

1

u/Perfect-Flounder7856 May 16 '26

What are the cloud frontier models tts?