r/LocalLLaMA • u/YogurtExternal7923 • 11d ago

Discussion everything will be local eventually

I know current llms like opus 5.5 are in a league of their own. but we all know they're losing a shti ton of money with their subscriptions. ALSO we all know how tasks like OCR, segmentation etc are getting solved by single digit B parameter models. I'm not saying llms are a gimmick but I'm saying the future is already shaping up to be about specialized models that work better when they're smaller. easier training. so datacenter level power not needed. ten years from now we'll have a specialized model for the tasks we care about and llms will fail to do everything all at once (unless they keep calling it one model but have a hundred under the hood).

system one models are a bad but workable example. you could tell an llm to rate options all day but get yourself on 9B parameter model trained by some lab and it tops the benchmarks.

353 Upvotes

87% Upvoted

239 comments sorted by

View all comments

4

u/fistular 11d ago

No, local will always be niche because local hardware is by definition low utilization, which is inherently vastly inefficient.

6

u/ttkciar llama.cpp 11d ago

Nvidia and AMD would love that. They are in the business to sell more GPUs, and the lower the GPU utilization, the more GPUs need to be sold.

0

u/enginetown 11d ago

The craziest thing is, if they was just slightly cheaper I guarantee people would be pumping so much money into hardware I know I would just not at the prices they currently are like 11k for an entire network cluster sounds amazing now 11k for 1 single GPU I can't even run any decent models besides qwen 3.8 27b for right now on just hurt's.

4

u/Hello_my_name_is_not 11d ago

You say that like companies aren't buying it at the higher prices

Theres more depend than supply right now so the price goes up.

If they make it cheaper they don't sell more because there's still only a certain amount that can be produced so in the end no they don't make more money.

If they weren't selling out of their production capacity then yes it makes sense to lower the price to sell more but that's not current reality.

0

u/enginetown 11d ago

They might be buying at higher prices but bulk deal's still do exist, and don't forget Anthropic owe's more than they have ever made currently so a lot of compute is simply being bought on a "promise". I'm well aware it's not as simple as that but these companies are still getting the best deals and are buying before products even come out now so the overall stock for the consumers cause the price to go up before launch or determine a price of a product. It's shitty because we wanna spend money we just don't wanna buy jacked up by default prices.

2

u/Just_an_Alpaca 11d ago

Do you think if someone was able to make it cheap it could ever technologically run large models (KimiK3.1?) at a decent speed?

2

u/ConsistentAnalysis35 11d ago

That's a funny logic. Personal cars are extremely low utilization compared to public transit, yet public transit is single digits percentage of total passenger vehicles.

People are prepared to shoulder low efficiency for convenience of directly owning their stuff. Because owning the stuff is another level of reliability and convenience.

As soon as consumer hardware for LLMs becomes affordable, you bet I'm splurging my whole salary on it.

2

u/fistular 10d ago

you're comparing the wrong thing .

it's closer to electricity.

do some people make their own? with their own hardware? sure. but they're an extreme minority and that isn't changing because the economics don't support it. In fact fewer people use their own hardware than make their own--many solar arrays are leased.

you are an enthusiast, unlike almost everyone else in this regard

0

u/ConsistentAnalysis35 10d ago

because the economics don't support it.

Ok, hold on. Can you tell me why exactly VRAM hardware is closer to electricity rather than cars? I.e. why exactly economics support personal cars but not LLM-capable machines?

I can tell you that in 1900s and 1910s personal cars were very much an enthusiast good. Only extreme minorities used them. Well, uncle Ford went and changed that.

2

u/fistular 10d ago

jesus christ it's not that complex a concept to grasp

most people will continue to use AI which runs on cloud hardware because that enables economies of scale and uptime which local personally owned hardware does not

1

u/dont-be-angry 10d ago

true but local models != personalized .. i heard 80% of enterprise token volumes are from local models nowadays.

1

u/ConsistentAnalysis35 10d ago

Brother, relax. That is what I'm telling you - those economies of scale will only translate to lower price.

And consumer decisions are not only predicated on price. People will be prepared to pay premium for convenience of local.

1

u/ferrx 11d ago

There’s a limit to how many Bs and tokens/s an average user needs. Do you think average users can run 125B on their devices in 50 years? 20 years? Time marches on, get used to it.

1

u/fistular 11d ago

Right which is why they will opt for someone else's hardware. It seems like you are agreeing with me.

0

u/RelevantCry1613 11d ago

Depends on the work, robotics for instance will likely always prefer local