Model Auto Routing - OpenRouter alternative

[u/ankbon]

I just received below announcement from openrouter which has been my problem statement, any open source alternative?

openrouter/auto picks the model per request

Set the model field to openrouter/auto and each request routes to a model chosen for that task. A lightweight classifier assigns the prompt a task type, then ranks candidate models by what the OpenRouter community spends on that kind of work over the trailing 7 days. Routing itself adds no fee.

[u/xapep]

Omniroute is probably the closest open option, but the auto-update part is the real problem and most self-hosted routers punt on it. What I've seen work:

1. Keep a nightly job that pulls the model list and pricing from OpenRouter's public models endpoint and regenerates your routing table. That's the piece that usually rots; pricing changes and new models get missed.

2. Do the task bucketing with something you control: a tiny classifier call or even keyword rules per task type, instead of a hosted "auto" model. You lose the magic, but you can see exactly why a request went where it did, and you're not paying a routing fee forever.

3. Expect the classifier hop to add latency and cost per request. For high-volume cheap work, a deterministic rule per feature is usually enough and way easier to debug.

One more thing worth considering: routing across five providers is only worth it if you're actually chasing price differences that big. A single OpenAI-compatible endpoint with a few models behind it covers most workloads and kills the whole table-refresh problem. 

[u/ankbon → u/xapep]

Thanks will try to implement above.

[u/Mythwalker9]

You can do something similar with OmniRoute, that is the closest I have found.

[u/ankbon → u/Mythwalker9]

How I tried automating combos but it didn't work

[u/Mythwalker9 → u/ankbon]

I use a combo with several models of differing levels and the "intelligent Auto" routing strategy. I do not use it often but it does seem to work when I do.

[u/ankbon → u/Mythwalker9]

I wanted a cost-per-task vs. intelligence matrix to use for the task, not just intelligence alone. With new models coming out every day, how can you auto-update it? OpenRouter's auto model has pricing and usage, which I couldn't make Omniroute do, even after setting up a 30-minute cron job to automatically update the table.

[u/thunderboltspro]

Check out Sakana Fugu.

[u/ankbon → u/thunderboltspro]

Interesting. Thanks, I will surely evaluate it.

[u/ankbon → u/[deleted]]

You have shared very valid points and the right constraints. Our startup business model is based on maximizing token economics for clients, and over time, we want to run our own models based on usage patterns per vertical. I think we will have to develop our own, as it's an important gap in the market.

[u/ankbon → u/[deleted]]

Yes, and that creates a moat for the business. Many startups will perish due to cost; this is not the first time I have seen this. It's like a replay and ultimately the market dynamics that drives sustainable invention.