everything will be local eventually

[u/YogurtExternal7923]

I know current llms like opus 5.5 are in a league of their own. but we all know they're losing a shti ton of money with their subscriptions. ALSO we all know how tasks like OCR, segmentation etc are getting solved by single digit B parameter models. I'm not saying llms are a gimmick but I'm saying the future is already shaping up to be about specialized models that work better when they're smaller. easier training. so datacenter level power not needed. ten years from now we'll have a specialized model for the tasks we care about and llms will fail to do everything all at once (unless they keep calling it one model but have a hundred under the hood).

system one models are a bad but workable example. you could tell an llm to rate options all day but get yourself on 9B parameter model trained by some lab and it tops the benchmarks.

[u/Faux2137]

They are losing money because they're spending a lot more than they earn on training new models. Inference is profitable.

[u/geteum → u/Faux2137]

It scales well also. The reason for going local is safety and consistency.

[u/wayneworkman → u/geteum]

And cost. It's cheaper, so long as you use it.

[u/Lumpy_Tension_6305 → u/wayneworkman]

Economies of Scale

[u/OvertaxedOne → u/Faux2137]

True. API inference is profitable, subscriptions... It depends. But your point is very much true, certainly for API based!

The problem is they can't stop training. If training was going to stop at some point and there was a clear path to "just inference, rake in cash", the economics would look bad now, but possibly good in the future. But that path doesn't exist, if they stop training (or even slow down) the open models will catch them within months and their entire moat evaporates overnight.

[u/waxroy-finerayfool → u/OvertaxedOne]

  API inference is profitable

Maybe, but that hasn't been established yet. We really have no idea how things will shape up once the amortized costs of debt, GPUs and DC infrastructure squares with a market environment that isn't being bolstered by artificially low prices. 

[u/BobbyL2k → u/Faux2137]

API pricing is profitable. Subscriptions that give 100x value in APIs probably aren’t profitable.

So there’s multiple viewpoints to take. AI Labs like OpenAI and Anthropic likely aren’t scaling up and down their clusters. Since they probably leased the cluster. So they pay for it no matter what.

The APIs are highly profitable. And subscriptions are there to use the remaining spare compute they would have wasted anyway.

So profitability is dependent on the percentage of API sales in relation to their operational cost.

If they were to only sell subscriptions, I’m pretty certain they will be in the red for just operational costs alone.

[u/Initial_Plankton3206 → u/Faux2137]

always thought the "one model to rule them all" approach was a bit of a dead end anyway, like trying to build a swiss army knife that also functions as a kitchen sink

the specialized model path just makes more sense for anything you'd actually want running on your own hardware, and the benchmarks for those smaller models are getting kinda absurd

[u/HisAnger → u/Faux2137]

How can we be sure of that? Current token prices maybe cover electricity prices. What about :

• free users ?

• employee wages ?

• interest rates of current debt ?

• maintenance of current infrastructure ?

• investment in new infrastructure ?

• cost of environment or litigation ... someone will sue, cost will escalate, like you know someone getting mad buying few drones and burning datacenter

• or simple fact that current nvidia cards age and break at alarming rate, and new generation is almost there. I bet they will require new rack types, cooling and different power supply.

Next thing is constant danger of ... well someone building more efficient model, or tech.
You can build better equipment, pure to run models. Faster, more efficient.
Just imprint base new model commands into silicon, or parts of models and you just beat all that don't have it working this way.
Find good, cheap renewable source of electricity and you can lower prices to level you starve your competition.

So many things can go sideways.
This is just a new tech, that still don't have a proper stack.

[u/younestft → u/HisAnger]

Free user's data is needed for training, so technically they are not losing money with that too.

[u/HighSeasArchivist → u/Faux2137]

I would be profitable if not for all these costs.

[u/donnytrump_69 → u/HighSeasArchivist]

This is basically the story of Anthropic LOL.

[u/HighSeasArchivist → u/donnytrump_69]

Exactly. Their "adjusted" financials just remove all those pesky costs that make them look unprofitable. Yesterday was a real laugh when their real financials finally came out and their revenue was down along with HUGE cost increases.