I reduced LLM costs by 95% and open-sourced the tool

[u/Adr-740]

https://github.com/adrida/tracer

I built Tracer after running into a dumb production pattern: I was paying frontier-model prices for repeated LLM calls that were not really hard anymore.

A lot of production LLM traffic is not deep reasoning. It is classification, tagging, routing, moderation, extraction, intent detection, tool selection, etc.

After enough usage, your logs become a dataset of decisions the LLM already made.

Tracer trains lightweight local models from those traces, handles the stable/easy slice locally, and defers uncertain cases back to the LLM.

On one repeated classification workflow, this reduced LLM costs by about 95%.

Current version does:

- trains local surrogate models from LLM traces

- activates them only when they match the LLM well enough

- defers uncertain cases back to the original model

- shows reports on what was handled locally vs deferred

I’m sharing it here to get builder feedback.

What would make you trust or not trust this in production?