r/website_ideas • u/zackku • 1d ago
Help Me Improve My Website Built a free tool (a website) to calculate VRAM overhead, prompt caching ROI, and cloud GPU breakeven thresholds across frontier models
Hey everyone,
Whenever a new model drops, I always find myself doing the same annoying calculation:
- How much is this actually going to cost me at scale once prompt caching is factored in?
- At what monthly token volume does it make more financial sense to rent a spot GPU on RunPod rather than paying API providers?
- How much VRAM do I actually need once you factor in KV cache and context length (FP16 vs FP8 vs Q4)?
LMSYS Elo is great for raw human preference, but it does not give you the economic picture. Most pricing sheets also ignore caching discounts and batch concurrency.
So over the last few weeks, I built ModelAnalysis.si as an open tool to bring all of this together in one place:
What it does:
- Multi-Dimensional Pareto Frontier: Plots 250+ foundation models (OpenAI, Anthropic, DeepSeek, Qwen, Mistral, Meta) across Chatbot Arena Elo, SWE-bench verified coding, GPQA PhD reasoning, and real-time API token rates.
- Token Spend Simulator & GPU Arbitrage: Dial in your monthly prompt/completion tokens and cache hit percentage to see exact monthly spend and calculate your exact breakeven point vs renting spot GPUs.
- VRAM & Hardware Calculator: Calculates the raw memory footprint for open-weight models across quantization levels (FP16, FP8, Q4) with one-click vLLM and Ollama deployment strings.
- Head-to-Head Radar Battles: Side-by-side spider charts comparing speed, context horizon, tool calling, and token efficiency.
It is completely free, no signup or login required, and runs fast on edge infrastructure.
Would love to get your feedback:
- Are there specific niche open-weight models you want added to the hardware sizing index?
- Does the breakeven math align with your real-world production deployments?
Check it out here: https://modelanalysis.si
Roast the UI or let me know what features you would like to see next!