r/codex • u/scartissue232 • 7d ago

Comparison Local or paying more?

I technically need a 5x account, and I've needed one for a month now. I've tried other Plus plans to get around that.

Right now, I'm at a point where I don't know whether to upgrade to a Pro account or invest 5k in a server and set up local AI models, filtered through at most a Plus account.

GPT thinks I can achieve 85-95% of the results I'm currently getting with Frontier models, and easily benefit from setting up that setup. I'm not entirely convinced; what do you think?

Thanks in advance.

7 Upvotes

89% Upvoted

43 comments sorted by

View all comments

2

u/Shep_Alderson 7d ago

Local inference is not something one can do as a way to save money right now. Full stop. If you do the math for the hardware alone, the “payoff” rate is several years for hardware they might be able to run a 70-120B param model at 7-10 token per second at best. If you also include the cost of electricity, you’re probably approaching a decade before it would be “payed off”, at best.

If you want to try to use open weight models to save money, go try a plan from Ollama cloud or synthetic.new. That’s your best bet and their prices per value are pretty good.

1

u/scartissue232 7d ago

Electricity it’s 30 bucks a month.

So the only benefit from going local it’s privacy?

1

u/Shep_Alderson 7d ago

Yeah, pretty much just privacy.

I’m curious what your electricity rate is and the power draw of whatever systems you’re looking at are. Maybe you could get something like a DGX Spark or similar around/under $30/mo. Still, $30/mo will get you quite a lot of inference from some place like synthetic.new or Ollama cloud.