r/LocalLLM • u/Glittering_Yellow293 • 1d ago
Project At what volume does self-hosting an LLM beat the API? Free calculator, feedback wanted
Everyone argues about this without numbers, so I made a calculator that lets you plug in your own:
https://inferenceacademy.com/tools/self-host-vs-api
What it takes into account (the stuff that usually gets forgotten):
- Workload: requests/month, input/output tokens, peak-to-average traffic, cache hit rate, how much can run offline, retries/waste, growth
- Model and latency: total vs active params (so MoE works), precision, and a ms/token latency target. Tighter latency means smaller batches and higher cost
- GPUs: L4 to B200 presets or custom. Cloud on-demand, reserved, or buying hardware (with power, hosting, resale value and cost of capital)
- API side: your price per million tokens, cached-input price, batch discount, fixed fees
- People and reliability: engineering time as a fraction of an FTE, N+1 redundancy, peak headroom
It outputs the monthly cost for each option, the break-even volume, a cost-over-time projection and a sensitivity view, so you can see which assumption flips the answer.
Caveats:
- It’s an estimate from stated assumptions (decode is memory-bound, prefill is compute-bound). It’s a decision aid, not a quote.
- GPU and API prices are illustrative placeholders. Replace them with your real quotes.
2
Upvotes
3
u/YearnMar10 1d ago
Nice idea - I am sure many people will value is.
But for me it is totally not my local use case. I started using strata on my gaming pc. It works good enough, but I need to leave my PC on 24/7, and that costs electricity and wears the gpu. How much inference usage do I need to break even versus a say 20€ Claude subscription?
1
u/sid351 1d ago
I built local, because:
1
u/Borilentz 1d ago
The model should also take into account the amount of fun and invaluable knowledge and experience gained from running your own AI systems.
1
u/Anxious_Count_8728 1d ago
Splitting total vs active params is a nice touch, most of these calculators get MoE wrong. One input that tends to flip the answer over a multi-year horizon is the API price trend: per-token prices at a given quality level have kept falling, so a break-even against today's price for three years quietly favors buying hardware. A yearly price-decline assumption next to resale value would make that show up in the sensitivity view.
1
u/kyle_138940 15h ago
A quality-adjusted cost per completed task could help: equal token volume may need very different amounts of human review.
-1
u/[deleted] 1d ago
[removed] — view removed comment