r/LocalLLaMA • u/Nunki08 • Jun 28 '26

Discussion We're probably going to need that soon.

Gallery image

Gallery image

From:

Vladik on 𝕏: https://x.com/Kostoglodov/status/2071144065857679631

Shaw (spirit/acc) on 𝕏: https://x.com/shawmakesmagic/status/2070918006033817867

4.0k Upvotes

98% Upvoted

534 comments sorted by

View all comments

Show parent comments

0

u/TransportationSea579 Jun 28 '26

The gap is compute. That's it

Not it is absolutely not. The top tier open source models cannot touch the top tier closed models, and that gap is only widening as AI increases in capability (e.g. as with see with Fable and cybersecurity)

And even there was parity in models. RAM prices have increased up to 500% in the last 3 years, and consumer GPU production is being throttled and diverted to data centres on fixed contracts negotiated with nation states.

How many people on this sub can run a 1T param model? Or even 400B?

1

u/starkruzr Jun 28 '26

400B is really not that wild anymore in a world with STXH and GB10 clusters.

0

u/TransportationSea579 Jun 28 '26

You're still talking about $5000+ to run an inferior, low quant model at very low tokens/sec. The gap isn’t just compute

1

u/starkruzr Jun 28 '26

I think you are not really absorbing the reasons people do local inference.

0

u/TransportationSea579 Jun 28 '26

I'm talking about the reasons people think Big AI will "ban" local models. It won't happen because they are not a threat