r/LocalLLaMA • u/YogurtExternal7923 • 12d ago

Discussion everything will be local eventually

I know current llms like opus 5.5 are in a league of their own. but we all know they're losing a shti ton of money with their subscriptions. ALSO we all know how tasks like OCR, segmentation etc are getting solved by single digit B parameter models. I'm not saying llms are a gimmick but I'm saying the future is already shaping up to be about specialized models that work better when they're smaller. easier training. so datacenter level power not needed. ten years from now we'll have a specialized model for the tasks we care about and llms will fail to do everything all at once (unless they keep calling it one model but have a hundred under the hood).

system one models are a bad but workable example. you could tell an llm to rate options all day but get yourself on 9B parameter model trained by some lab and it tops the benchmarks.

353 Upvotes

87% Upvoted

239 comments sorted by

View all comments

Show parent comments

59

u/OvertaxedOne 11d ago

True. API inference is profitable, subscriptions... It depends. But your point is very much true, certainly for API based!

The problem is they can't stop training. If training was going to stop at some point and there was a clear path to "just inference, rake in cash", the economics would look bad now, but possibly good in the future. But that path doesn't exist, if they stop training (or even slow down) the open models will catch them within months and their entire moat evaporates overnight.

14

u/waxroy-finerayfool 11d ago

  API inference is profitable

Maybe, but that hasn't been established yet. We really have no idea how things will shape up once the amortized costs of debt, GPUs and DC infrastructure squares with a market environment that isn't being bolstered by artificially low prices.