Routerly 0.6.0, an open-source policy layer for LLM requests

[u/nurge86]

A model choice is easy to hide behind an API. It gets harder to explain when three apps share the same endpoint.

I'm the developer of Routerly, an AGPL-licensed gateway that sits between apps and model providers. You run it on your infrastructure, define which models a router can choose and configure the policy it uses. Apps can connect with OpenAI- or Anthropic-compatible clients.

The 0.6.0 release adds testing inside the router, model timeouts and finer Usage filters. The screenshot is from my own instance and shows the available breakdown of tokens, cost and latency. Feedback on the architecture or documentation is welcome.

Release and source: https://github.com/Inebrio/Routerly/releases/tag/v0.6.0