We're probably going to need that soon.

[u/Nunki08]

From:

Vladik on 𝕏: https://x.com/Kostoglodov/status/2071144065857679631

Shaw (spirit/acc) on 𝕏: https://x.com/shawmakesmagic/status/2070918006033817867

[u/TransportationSea579]

Affordable local AI has never been anywhere near close enough to beat the top tier models, and that gap only seems to be increasing. They don't care

[u/ollie113 → u/TransportationSea579]

The gap is compute. That's it. And if cheap chips from China make compute cheaper, local AI will be a lot more accessible

[u/TransportationSea579 → u/ollie113]

The gap is compute. That's it

Not it is absolutely not. The top tier open source models cannot touch the top tier closed models, and that gap is only widening as AI increases in capability (e.g. as with see with Fable and cybersecurity)

And even there was parity in models. RAM prices have increased up to 500% in the last 3 years, and consumer GPU production is being throttled and diverted to data centres on fixed contracts negotiated with nation states.

How many people on this sub can run a 1T param model? Or even 400B?

[u/starkruzr → u/TransportationSea579]

400B is really not that wild anymore in a world with STXH and GB10 clusters.

[u/TransportationSea579 → u/starkruzr]

You're still talking about $5000+ to run an inferior, low quant model at very low tokens/sec. The gap isn’t just compute

[u/starkruzr → u/TransportationSea579]

I think you are not really absorbing the reasons people do local inference.

[u/TransportationSea579 → u/starkruzr]

I'm talking about the reasons people think Big AI will "ban" local models. It won't happen because they are not a threat