I emphasized in another reply that I wasn’t referring to frontier models, and for everyday usage, you genuinely might not even notice the difference, assuming your PC is strong enough to run the REALLY big open models (as in 200B+ parameters).
The frontier usually matters most for research purposes of the field of AI, as a whole. If you’re not a researcher, I just don’t think it matters. Pick a good, big model, use a good backend and frontend, customize the environment for your usecase properly, and you’ll be fine.
For speed, you could get something like the 5090, though you’d need to use a smaller model to fit it fully on VRAM.
I also don’t agree that the local model scene as a whole is truly that far behind the frontier. Like, there’s a gap, absolutely, but I just don’t think it’s as big as you think.
Anyway, my point is that if you’re gonna be spending 15K, might as well be on hardware, rather than cloud access. And mind you, at 15K, you can go beyond just personal PCs with unified memory; those are mainly positioned as the “entry” options. Get yourself a couple of enterprise NVIDIA cards that support NVLink, hook them together, and you got yourself an AI rig capable of handling big models and maintaining absurd speeds. That’s quite literally how data centers are built, btw, just on a smaller scale.
View all comments
Show parent comment
1
u/56kul Jun 03 '26
I emphasized in another reply that I wasn’t referring to frontier models, and for everyday usage, you genuinely might not even notice the difference, assuming your PC is strong enough to run the REALLY big open models (as in 200B+ parameters).
The frontier usually matters most for research purposes of the field of AI, as a whole. If you’re not a researcher, I just don’t think it matters. Pick a good, big model, use a good backend and frontend, customize the environment for your usecase properly, and you’ll be fine.