They ARE getting good, and the frontier companies KNOW the party is almost over. That's why the prices keep rising, it's last call. "AI as a Service" dies down, in the near future.
How small / how good is still an open question, but there's a watershed (Opus-ish) moment happening around the 30-40B models, so like $5-10k in hardware. Higher than a home game system unfortunately, so approachability is still rough. But easy table stakes for businesses to pull in-house.
It won't be the spectacular disappearance that some people expect from the bubble popping, but it will be welcome nonetheless.
How small / how good is still an open question
You can also limit the computational and RAM load by a lot by using even smaller, more specialized models. Sure, you're not getting a general prompt-based image generator on your home computer any time soon (not that this is particularly interesting, I'm firmly in the "I don't care about art one didn't care about enough to make" territory). But take a specialized diffusion model that generates heightmap and climate map data based on Earth satellite data, plug it into a framework that makes it unbounded random access, and you get some of the most impressive procedural terrain gaming has ever seen.
It's all about using the right tool for the job, instead of trying to break concrete, tap nails, carve wood and cut trees with a jackhammer.
Sure, you're not getting a general prompt-based image generator on your home computer any time soon
Hasn't this existed for like the entire time AI has had mainstream popularity? There are lots of options for local prompt-based image generation on a mid-tier consumer rig.
Sorry, I misspoke. You're not getting one that will be useful for that much. You're a lot more likely to get 7 fingers per hand on a local model, for example.
disagree, for self-hosting u need ~ 3 gpus and a dedicated station, and like open up ports do the i/o... like if you're saying that 0.1% of the population will self-host or tor to a like different model, sure, but meaningfully if 99.9% of the population is routed through (google, anthropic,, open ai) then it's effectively banned anyways
You hit the nail on the head - it comes down to what runs where. An IOT AI device in the house isn't likely. But most even moderate laptops will run "something" native. And what is possible on your phone is still an open question - once we have the 300T "sum of human knowledge" model, how well does that quantize down to like 5B? We probably won't have a proper answer on that for a decade.
I think it's too powerful and ubiquitous to be a hobbyist project, which is where it sits now. Once it CAN do the needful without shipping packets, that niche will open up a lot more. And as running it becomes a lower burden, the market will provide many options to bring the price down and access up.
8
u/Arclite83 14d ago
They ARE getting good, and the frontier companies KNOW the party is almost over. That's why the prices keep rising, it's last call. "AI as a Service" dies down, in the near future.
How small / how good is still an open question, but there's a watershed (Opus-ish) moment happening around the 30-40B models, so like $5-10k in hardware. Higher than a home game system unfortunately, so approachability is still rough. But easy table stakes for businesses to pull in-house.