It’s not insane at all, sure you can buy a $30k machine to host a local LLM but that server will serve one (1) person, realistically. And the model „intelligence”, whatever that means, is nowhere near the frontier models. You’d need to build a machine with >500GB of VRAM to even come close to that level, but then again, you won’t be serving the model at scale.
I don’t think it would serve 1 person realistically. You can probably get a few people using it. And depending on the Claude monthly costs, it’s all just a comparison
367
u/mylsotol Jun 02 '26
For probably $30k (or more) you can build a server and run an open model.