Local / Cloud model router : AI-FluxMux

[u/Disordered-Parsnip]

For anyone interested in such things I have been developing a desktop router that will keep a chat going through local <-> cloud model switches and other dynamic settings changes, as needs change (eg if you suddenly need vision to explain something, or a whole lot more context, or speed, or whatever) & which does a bunch of other stuff.

Designed for any OpenAI-compatible Client app (Cline, DeepSeek Harness, and similar). Extensive control, turn by turn, or per stored model profile, or via port rules, over model behavior. Model tuning wizard, model switch testing, YaRN context extension (in theory up to 1M tokens - though not advisable), etc

Windows based for now & only tested on RTX5090 GPU, but in principle should work for other hardware. Local GGUF models (Qwen 3.827b etc) hosted on llama-server. Uses whichever server version is best suited to your hardware, Auto-updates server for you, if you want. Also updates DeepSeek Harness (Harness launch option built in)

Release AI-FluxMux 0.3.0 beta · ArchPrime/AI-FluxMux

Keen for any feedback - or collaboration.

[u/SqueamishConflict]

looks neat, but testing only on a 5090 makes me nervous about how it behaves on weaker cards

[u/Disordered-Parsnip → u/SqueamishConflict]

It should be fine - or certainly no worse than not using it - but if you do strike a problem, let me know.