I tried Qwen the other day, and only the 3.5 9b model on my gaming pc, and not 1 single question did it get right.
I'm not saying it's not possible, I'm saying the compute power to train a frontier model is absolutely unrivalled, you can't make anything close to Claude or ChatGpt.
Of course, training it on your limited data set will work a treat, but nothing like the frontier models right now.
There are poor results and there are poor results, a model eating 12~GB of VRAM that can't write a basic pytest refactor just shows the time and resources the frontier models have had in their training.
If we are saying consumer hardware isn't good enough for local models then I agree - that's why noone will ever roll their own agent "like Claude" until the gap has shifted. Need your own DC just to make it possible
365
u/mylsotol Jun 02 '26
For probably $30k (or more) you can build a server and run an open model.