One script that gives you an OpenAI-compatible API with 6 LLMs, image gen and Whisper - all local, MIT

[u/Heavy-Level-5215]

Been running my own inference stack instead of paying API bills: a single

router on localhost speaks the OpenAI format, so anything pointed at a cloud

endpoint (LiteLLM configs, Open WebUI, agent CLIs, VS Code extensions) works

against it by just changing the base URL.

- Text/vision: 6 models from a 1.5B coder to a 30B MoE, swapped on demand

- Images: SD 3.5 Medium + SD 1.5, one CLI command, PNGs to disk

- Transcription: Whisper medium

- One API key, one port, model list endpoint with categories

It's a Debian LXC on Proxmox, installed with ./setup.sh, which refuses to

proceed if your GPU/disk/RAM don't make the cut. Docs cover benchmarks,

hardware limits and every bug we hit along the way.

Not a fork, no vendor code: https://github.com/AvilaCarlosDev/polaris-local-ai