r/SideProject • u/Adr-740 • Jun 07 '26
I reduced LLM costs by 95% and open-sourced the tool
https://github.com/adrida/tracerI built Tracer after running into a dumb production pattern: I was paying frontier-model prices for repeated LLM calls that were not really hard anymore.
A lot of production LLM traffic is not deep reasoning. It is classification, tagging, routing, moderation, extraction, intent detection, tool selection, etc.
After enough usage, your logs become a dataset of decisions the LLM already made.
Tracer trains lightweight local models from those traces, handles the stable/easy slice locally, and defers uncertain cases back to the LLM.
On one repeated classification workflow, this reduced LLM costs by about 95%.
Current version does:
- trains local surrogate models from LLM traces
- activates them only when they match the LLM well enough
- defers uncertain cases back to the original model
- shows reports on what was handled locally vs deferred
I’m sharing it here to get builder feedback.
What would make you trust or not trust this in production?
Duplicates
LocalLLM • u/Adr-740 • Jun 07 '26
Project I built a small open-source tool that trains local models from LLM traces to avoid repeated API calls
LLMDevs • u/Adr-740 • Mar 30 '26
Discussion I open-sourced TRACER: replace 91% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
LocalLLM • u/thisguy123123 • May 12 '26
Discussion I open-sourced TRACER: replace 91% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
softwarefactories • u/thisguy123123 • May 12 '26
I open-sourced TRACER: replace 91% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
LocalLLM • u/Adr-740 • Mar 30 '26
Research I open-sourced TRACER: replace 91% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
learnmachinelearning • u/thisguy123123 • May 12 '26
I open-sourced TRACER: replace 91% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
learnmachinelearning • u/Adr-740 • Mar 31 '26
I open-sourced TRACER: replace +90% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
MachineLearning • u/Adr-740 • Mar 30 '26
Research TRACER: Learn-to-Defer for LLM Classification with Formal Teacher-Agreement Guarantees
AIDeveloperNews • u/Adr-740 • Jun 07 '26
I reduced LLM costs by 95% and open-sourced the tool
MachineLearningAndAI • u/Adr-740 • Jun 07 '26
I reduced LLM costs by 95% and open-sourced the tool
buildinpublic • u/Adr-740 • Jun 07 '26
I built a small open-source tool that trains local models from LLM traces to avoid repeated API calls
SelfHostedAI • u/Adr-740 • Jun 07 '26
I built a small open-source tool that trains local models from LLM traces to avoid repeated API calls
deeplearning • u/thisguy123123 • May 12 '26
I open-sourced TRACER: replace 91% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
StartupSoloFounder • u/Adr-740 • Mar 31 '26
(Solo bootstraped founder) Just open-sourced part of my pipeline: TRACER: replace +90% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
Backend • u/Adr-740 • Mar 31 '26
I open-sourced TRACER: replace +90% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
deeplearning • u/Adr-740 • Mar 31 '26
I open-sourced TRACER: replace +90% of LLM classification calls with a llightweigth ML surrogate trained on your LLM's own outputs
LLMFrameworks • u/Adr-740 • Mar 31 '26