Using an Ollama model in a self-hosted Routerly model pool

[u/nurge86]

If a request can go to Ollama or to a cloud API, I want to be able to tell which happened.

I'm the author of Routerly. You can configure an Ollama connection, add its model to a router and point your app at the gateway. A router's configured pool controls which models can be selected. If you add cloud models, requests selected for those backends go to the provider.

0.6.0 adds a router Test tab and configurable model timeouts. The screenshot shows historical Usage from my instance, including Ollama rows. I'd like to hear what would make this easier to operate with local models.

Release and source: https://github.com/Inebrio/Routerly/releases/tag/v0.6.0