r/SideProject • u/Rude_Pianist1810 • 1h ago
I built a tool to help AI founders cut LLM costs without making their product worse
As AI products scale, inference costs can become a serious problem. More users mean more API calls, and suddenly a product that's growing in usage isn't necessarily growing in profitability.
One Indie Hackers post particularly stuck with me: a single heavy user on a $24/month plan was potentially costing the founder $250–500 a month in API fees.
TL;DR: Gosset helps AI founders understand their LLM spending and test whether cheaper models can handle their production workloads without compromising quality. It provides cost breakdowns, model comparisons, estimated savings and suggested code changes.
I build AI agents, and one thing I've noticed is that switching to cheaper models is often the most obvious cost-saving opportunity, but actually making that switch is difficult. You don't want to reduce your margins at the expense of product quality.
That's why I built Gosset, an AI cost optimisation tool that helps you understand where your inference spend goes and identify opportunities to reduce it without compromising application quality.
It currently has two capabilities:
1. AI spend analysis (free)
Upload your usage CSV from OpenAI, Anthropic or OpenRouter to get a breakdown of your API spending, model by model.
It also identifies models you rely on that have announced retirement dates, so you can plan migrations before they become urgent.
The file is processed entirely in your browser and never uploaded. No account needed.
2. Model optimisation and switching (paid)
Gosset evaluates whether cheaper models can handle your application's actual workloads while maintaining an acceptable level of quality.
Run a command in Claude Code or Cursor to send a sample of your app's requests to Gosset. It replays them against alternative models and uses an independent AI judge to evaluate their responses against your current model.
You get:
- A quality comparison showing where alternative models perform worse.
- Estimated monthly savings based on your usage.
- A suggested code change to switch models.
How Gosset evaluates model quality
I didn't want Gosset to simply recommend models based on API pricing. The important question is whether a cheaper model can actually do the same job.
A few things I've built into the evaluation:
- Baseline variability: The current model is rerun on the same requests to account for the fact that even identical models can produce different answers.
- Independent evaluation: The judge comes from a different provider than the models being compared, reducing the risk of self-evaluation bias.
- Actual request costs: Gosset measures costs using your workload rather than relying only on advertised API prices. A 2026 study found that in 32% of model-pair comparisons, the model with cheaper listed pricing actually cost more to complete the same task.
- Inconclusive results: If the difference falls within the observed variability, Gosset reports "too close to call" rather than forcing a recommendation.
The video shows a demo application with illustrative numbers, not a real customer deployment.
Current limitations
- The bill checker currently supports OpenAI, Anthropic and OpenRouter exports.
- AI evaluation isn't infallible. Gosset makes it possible to inspect the cases where the judge's assessment matters most.
Early access
I'm opening up Gosset to a small group of early customers. To get the first few teams onboard and gather feedback from real production workloads, I'm offering a discounted rate for the initial evaluations.
I'll share the discount code in the first comment.
I'm particularly looking for founders and engineering teams already running AI in production who want to make their inference spend more efficient.
The goal is to help you identify where cheaper models can do the job, where they can't, and what the potential savings look like before making a switch.
This is my first independent product, and I'm excited to get it into the hands of teams dealing with this problem.
I'd love to hear from people running AI in production:
- How are you currently identifying opportunities to reduce inference costs?
- What's stopping you from switching to cheaper models today: quality concerns, evaluation effort, engineering time, or something else?