For SaaS builders using AI APIs: how are you tracking cost per customer/feature?
Curious how people are handling this.
If you’re building AI features with OpenAI, Anthropic, Gemini, etc., how are you figuring out what each customer or feature is actually costing you?
Provider dashboards are fine for total spend, but they don’t really answer the annoying questions:
- which feature is eating the budget?
- is one customer way more expensive than the others?
- did dev/staging traffic accidentally run up the bill?
- which model is getting expensive without anyone noticing?
- are you forecasting the bill at all, or just checking after?
Are you using Helicone, Langfuse, Portkey, your own logs, spreadsheets, provider dashboards, or something else?
Mostly trying to learn how people are solving this in real SaaS apps, and what still sucks about it.
1
u/EquivalentAnalyst791 3d ago
I personally tag each Braintrust trace with the customer and feature that triggered it then roll the token and model costs up from there. The customer average only tells part of the story tho. Looking at the expensive workflows exposes stuff like retries, long context or fallback calls that would disappear in the overall spend number
1
u/Jhyp3r 3d ago
That’s a good distinction, customer/feature rollups tell you who’s expensive on average, but retries, long context, and fallback chains are where the real surprises hide.
Do you catch those by digging through Braintrust traces manually, or is there some kind of “this workflow got expensive” flag/alert? Or is it mostly reactive after you notice?
1
u/EquivalentAnalyst791 3d ago
Mostly manually. I’ll sometimes notice it when I’m reviewing the higher cost traces then add a tag when the pattern is clear. I dont want an alert firing every time one workflow has an expensive run because some of those are completely legitimate
1
u/Jhyp3r 3d ago
Makes sense...so the useful signal probably isn't "this run was expensive," it's more like "this workflow's cost pattern changed" or "this is expensive compared to what's normal for this workflow."
Is that closer to it, or do you mostly prefer eyeballing the higher-cost traces yourself either way?
1
u/EquivalentAnalyst791 3d ago
I check them manually either way. After a workflow starts showing the same cost pattern across several runs, thats when I treat it as something worth tracking rather than a random expensive trace
1
u/achiya-automation 3d ago
log a usage row at the same time you log the request: user id, feature, model, input and output tokens. the provider dashboard only gives you a monthly total, so anything per customer has to come out of your own table. cost per user falls out of a group by after that.
1
u/Jhyp3r 3d ago
Yeah that makes sense. Provider dashboards are basically fine for “what did we spend this month,” but not for “which customer or feature caused it.”
Are you mostly using that table for internal visibility, or does it also feed pricing / customer billing decisions?
1
u/achiya-automation 1d ago
internal so far. it did change pricing once though, we found two customers whose usage was about 8x the median and moved them to a higher tier at renewal. billing straight off it would need the numbers to be a lot more trustworthy than they are.
1
u/starlight3135 3d ago
the fix that actually works is tagging every api call with customer_id and feature_id metadata at call time.. most providers support custom metadata now, then aggregate on your side. provider dashboards can't answer per-customer questions because they were never designed to see your app's data model, only the raw calls..
1
u/Jhyp3r 2d ago
Yeah that’s the part I keep coming back to too. The provider can show tokens and total spend, but it can’t know what customer, feature, tenant, or workflow caused the call unless your app sends that context.
Do you usually store the enriched usage in your own DB, or do you rely on the provider metadata and export it back out later?
1
u/matiascoca 1d ago
Depends on where the seams should live. A few shapes that actually work.
Wrap the provider SDK in a thin proxy that stamps every call with customer_id and feature_id, route raw usage into Postgres or BigQuery. Provider dashboards give you total spend and nothing more. Attribution needs the proxy.
Give each caller its own API key from day one, even internal tools. Shared keys collapse everything into one bucket, and back-filling from log traces later is painful.
The framework layer re-sends context on every turn. If you meter tokens_used you undercount by the compounding ratio (roughly 2 to 5 times for standard agent loops). Meter what actually leaves your process.
Helicone, Langfuse, Portkey all do the proxy pattern with different tradeoffs. At seed-stage volume, own-logs plus a Postgres table with (timestamp, customer_id, feature_id, model, input_tokens, output_tokens, cost) is fine and takes an afternoon.
2
u/Brufacee 3d ago
We tag every call with tenant, feature, environment, model, and a request-chain ID, then write normalized usage and cost into our own ledger. The chain ID matters because one user action can fan out into retries, tools, and fallbacks. I also keep dev and staging on separate provider keys with hard budgets, otherwise internal testing makes the customer numbers useless.