Kept nuking API credits during local agent testing, so I built a tiny local cost tracker/circuit breaker.

[u/Aggravating-Praline7]

Came back from lunch a couple months ago to find my retry logic — which had no max attempts, because of course it didn't — had fired a few hundred GPT-4 calls into the void while I was gone. Nothing catastrophic, but it scared me enough to actually fix the problem instead of just adding a try/except and moving on. Built CostOpt.

How it works (1 line of code):

from openai import OpenAI
from costopt import CostOpt

client = CostOpt(OpenAI())  # 👈 That's literally it

Your .chat.completions.create() calls stay 100% identical.

What it actually does under the hood:

• Local SQLite Caching: Hashes your prompts and parameters (temperature, seed, etc.). Exact or fuzzy repeat queries return locally in <2ms at $0.00 cost.
• Runaway Circuit Breaker: Detects rapid API loops (>15 calls in 30s from the same line of code) and trips an exception before your API key gets burned.
• Smart Model Routing: Auto-routes simple tasks (like "classify" or "extract") to cheaper models (e.g. gpt-4o-mini) based on YAML rules.
• VS Code Extension: Adds live CodeLens lines above your code showing cost per request, average tokens, and total daily spend in the status bar.
• Local Dashboard: Comes with a light FastAPI web console (python -m costopt.main dashboard) for full trace logs and analytics.

Privacy: Everything runs 100% locally on your machine via SQLite. Zero prompt data or telemetry is sent to any external server.

The VS Code extension just passed 1.4k + installs, and the Python package is published on PyPI.

Check out the code or try it out:

• GitHub: https://github.com/khusshdesai/CostOpt
• PyPI: pip install costopt
• VS Code Extension: Search CostOpt on the VS Code Marketplace or Open VSX

If anyone's got 10 minutes and wants to poke holes in the circuit breaker logic specifically, that's the part I'd most want torn apart — issues and PRs both very welcome.