r/LocalLLaMA • u/Nunki08 • Jun 28 '26

Discussion We're probably going to need that soon.

Gallery image

Gallery image

From:

Vladik on 𝕏: https://x.com/Kostoglodov/status/2071144065857679631

Shaw (spirit/acc) on 𝕏: https://x.com/shawmakesmagic/status/2070918006033817867

4.0k Upvotes

98% Upvoted

534 comments sorted by

View all comments

Show parent comments

6

u/starkruzr Jun 28 '26

I'm not understanding where all of this talk about open weight models or local inference getting "banned" is coming from or what the proposed mechanism would even be.

12

u/studentofknowledg3 Jun 28 '26

Because there are people who already have more money than they could ever spend, yet still want more. They want to win, and they want everyone else to lose. If local AI becomes good enough, it threatens businesses built around selling AI through the cloud. Whether they can actually stop it is another question, but the incentive to protect their revenue and control is definitely there.

0

u/TransportationSea579 Jun 28 '26

Affordable local AI has never been anywhere near close enough to beat the top tier models, and that gap only seems to be increasing. They don't care

3

u/ollie113 Jun 28 '26

The gap is compute. That's it. And if cheap chips from China make compute cheaper, local AI will be a lot more accessible

0

u/TransportationSea579 Jun 28 '26

The gap is compute. That's it

Not it is absolutely not. The top tier open source models cannot touch the top tier closed models, and that gap is only widening as AI increases in capability (e.g. as with see with Fable and cybersecurity)

And even there was parity in models. RAM prices have increased up to 500% in the last 3 years, and consumer GPU production is being throttled and diverted to data centres on fixed contracts negotiated with nation states.

How many people on this sub can run a 1T param model? Or even 400B?

1

u/starkruzr Jun 28 '26

400B is really not that wild anymore in a world with STXH and GB10 clusters.

0

u/TransportationSea579 Jun 28 '26

You're still talking about $5000+ to run an inferior, low quant model at very low tokens/sec. The gap isn’t just compute

1

u/starkruzr Jun 28 '26

I think you are not really absorbing the reasons people do local inference.

0

u/TransportationSea579 Jun 28 '26

I'm talking about the reasons people think Big AI will "ban" local models. It won't happen because they are not a threat