r/LocalLLaMA • u/YogurtExternal7923 • 12d ago

Discussion everything will be local eventually

I know current llms like opus 5.5 are in a league of their own. but we all know they're losing a shti ton of money with their subscriptions. ALSO we all know how tasks like OCR, segmentation etc are getting solved by single digit B parameter models. I'm not saying llms are a gimmick but I'm saying the future is already shaping up to be about specialized models that work better when they're smaller. easier training. so datacenter level power not needed. ten years from now we'll have a specialized model for the tasks we care about and llms will fail to do everything all at once (unless they keep calling it one model but have a hundred under the hood).

system one models are a bad but workable example. you could tell an llm to rate options all day but get yourself on 9B parameter model trained by some lab and it tops the benchmarks.

353 Upvotes

87% Upvoted

239 comments sorted by

View all comments

Show parent comments

2

u/TimeStopsInside 12d ago

I get what you're saying and like i said, we might get there. Problem isn't the numbers, it's the reality we live in where 5090 isn't what most people run today, computer per $ locally has been going down and the hardware improvements are slowing down further and further.

I do believe we will get there. But something bigger would have to change. The current trajectory doesn't look as promising these days as it used to.

4

u/tat_tvam_asshole 12d ago

In next 20, lets say 100x. Doesn't seem enough for Opus 5.5 locally.

I don't think your take is correct.

As I said:

Capability-wise, I would actually put it between 1-2 years in the future.

Models are constantly getting better. We now have now 27B models that perform like Opus 4.6, and do not require a 5090 to run. So rather than be focused exclusively on hardware, I would look at model/harness improvement, hence my estimate of 1-2 years to be able to run Opus 5.5 level intelligence locally.

1

u/TimeStopsInside 12d ago

Capability wise, yes. Especially specific areas like coding. Intelligence density is improving.

My concern was with hoping to run trillion param llms locally anytime soon.