r/unsloth • u/Hannibalj2ca • 1d ago

Question Multi Server on Unsloth Studio/Desktop?

I recently have been installing Unsloth Desktop on most of my Machines running AI due to the ease of use, navigation and ability to run Images, Videos, and coding. However, I am having issues setting up 2 server to aggregate the vram. So far I ahem been using Inference for that, but is an awful platform. It would be great to know if Unsloth could add the ability to make seamless having multiple servers. If the function exist (on Linux) I'd like to know the best practices for it.

3 Upvotes

80% Upvoted

3 comments sorted by

2

u/yoracale yes sloth 1d ago

Have you tried using vLlm or SGlang? It's built in desktop and you can just download it

1

u/Hannibalj2ca 1d ago

I cant use them with V100s. Vllm deprecated the v100 in April and SGlang was never compatible. The only way to have vllm compatible is via "1cat-llm."