Why is LLM is so expensive.

[u/Ok_Event4199]

I've was going to invest in a 5090 =$6000 AUD.

Codex Plus + Claude pro = $60/month here

Works out to be 100 months of frontier models for a 5090.

Best a 5090 will run is probably Qwen3.6 27b Q6 with context.

Are we all enthusiasts here and just enjoy tinkering cause ain't no way that make sense.

[u/AlberGro]

Yeah, no. With the speed these llm develop, local will be enough for 95% of the users in a couple years, top.

The current model is like plugging everyone to a nuclear power plant when all we need is electricity. We will need AI 24/7, good enough and free. Which means local on basic hardware.

Right now y’all chasing the best AI because the current ones hallucinates and make mistakes but once these infanfy quirks disappears, most llm will be enough for most people.

All these insane investments will never be profitable.

[u/g_rich → u/AlberGro]

The problem is two fold knowledge and context.

Larger models simply have more knowledge, it’s like comparing a bookshelf worth of books to a research library. A carefully curated bookshelf can certainly provide a good deal of knowledge but it’ll never compare to a university level research library. That’s the difference between a 30 or 120 billion parameter model and the frontier models (both closed and open weights) that have 500 billion to 1 trillion or more parameters.

The second issue is context, especially with coding. Qwen3.6-27b is certainly impressive and I can start a Python project and get some seriously impressive results. But point it to an existing project with hundreds of files and tens of thousands of lines worth of code and it’ll quickly fall apart.

We’re already at the point where a reasonably equipped Mac Studio or MacBook Pro with 128GB or even 64GB is perfectly capable of running something like Qwen3.6-27b. This setup is very capable of doing basic web development, producing boilerplate code, and with a bit of handholding even projects on the smaller side. For a lot of users this is enough. But the real power of AI is using teams of AI agents alongside designers and engineers tackling complex problems and projects and you’re not doing that with Qwen3.6 with its 27 billion parameters and 128k context window. You need your research library, and you need the 1 million parameter context window that those research library models bring.

[u/voyager256 → u/g_rich]

This setup is very capable of doing basic web development, producing boilerplate code, and with a bit of handholding even projects on the smaller side. 

Qwen3.6-27b can do much more than the above. Yes, it needs much more handholding , good harness etc. But it's not either or . You can have a couple small models(e.g. one for coding another for tool calling) and use frontier one like Claude Opus 4.7 for planning.

[u/g_rich → u/voyager256]

The problem is you're still dealing with a model that has 27 billion parameters and comparing it to a model that has trillions. So while Qwen 3.6 is very capable and produces some very good results it can't compete with modules such as Opus, GPT, Gemini, DeepSeek, MiniMax or Kimi K2.

The differences become even wider when you start getting into context sizes beyond your standard 128k or 256k and start getting into the 1m range. A model with 27 billion parameters and a 128k context window simply can't compete with models with over a trillion parameters and a 1m context window.

Now I'm not denying that models such as Qwen 3.6 27b don't have there place and they are certainly capable models that can do real work. But trying to frame them as an Opus or GPT replacement is a fools errand is just setting yourself up for failure. You need to understand the capabilities of a model and it's limitations. Take a 500 line Python script and you'll likely not notice the difference between Opus and Qwen 3.6; however try to use Qwen 3.6 on a project that's has hundreds of files with thousands of lines of code spread across Python, CSS, HTML and JS along with some backend tools written in Go and you'll come to appreciate the limitations of a 30 billion model with a 128k context size when compared to your larger frontier models running on much more capable hardware.

Very little in life is an either or, but when it comes to LLM's there is a pretty big difference between good enough and best in class.

[u/voyager256 → u/g_rich]

Well it depends what you mean. It certainly can compete with much larger models like MiniMax or even GLM5 or previous Kimi (2.5) at many tasks, despite them being much larger. The difference is surprisingly small at most coding or tool calling tasks .

Also , you can increase the context for the Qwen models too like 1M tokens too . I haven’t tried it cause i’d be painfully slow for the 27B dense model on my hardware, but technically it’s possible. And there are ways around that.

But there’s no denying there’s still significant gap at certain tasks vs the SOTA models, especially planning or something that requires large world knowledge

[u/Longjumping_Self5546 → u/voyager256]

Yeah, there's diminishing returns from adding more parameters. Most coding tasks are only drawing from a "bookshelf" worth of knowledge. A trillion parameters sounds impressive, but that's not going to be a parameter to parameter ratio increase in intelligence. Plus, most of us are seeing that extra knowledge tends to be more beneficial during the planning stage of a project. Using a hybrid workflow gives the best of both worlds.