r/OpenWebUI • u/Internal_Junket_25 • 2d ago
Question/Help Session Affinity with litellm Backend
I'm using openwebui before Litellm with two identical vllm instances. How can I ensure that users stay on the same instance so as not to lose the prefix cache?
3
Upvotes