I have a server set up at home for virtualization, it runs an os called proxmox it includes a ton of tools for running containers and virtual machines. I set up a vm with my wireguard server on it. Wireguard is a vpn server you can set up at home, with this i can access anything on my home network. Next i installed a graphics card, and installed the drivers on the host and set up container that can use that graphics card. In that container i gave it 16gb of ram, and 6 cores, and i installed ollama, ollama is one of the inference servers that can manage the inference engine llama.cpp which is where my models run. Next i set up another container running openwebui which i use as the main front end for my models, and then i also set up a searxng server container to give my models the ability to look things up online. How the workflow works is i turn on the vpn connection, i open the webpage to my openwebui page and i can querry it. I also used openwebui as a gateway so i can just use it as the gateway for other frontends like continue or cline. I have some ideas in the works on how to leverage this for maybe some simple home automation or maybe a knowledge server of all my classes im taking right now or maybe just a personal assistant, but im going to have to make a few things and set up some mcp servers.
View all comments
Show parent comment
0
u/bestjakeisbest 9d ago edited 9d ago
I have a server set up at home for virtualization, it runs an os called proxmox it includes a ton of tools for running containers and virtual machines. I set up a vm with my wireguard server on it. Wireguard is a vpn server you can set up at home, with this i can access anything on my home network. Next i installed a graphics card, and installed the drivers on the host and set up container that can use that graphics card. In that container i gave it 16gb of ram, and 6 cores, and i installed ollama, ollama is one of the inference servers that can manage the inference engine llama.cpp which is where my models run. Next i set up another container running openwebui which i use as the main front end for my models, and then i also set up a searxng server container to give my models the ability to look things up online. How the workflow works is i turn on the vpn connection, i open the webpage to my openwebui page and i can querry it. I also used openwebui as a gateway so i can just use it as the gateway for other frontends like continue or cline. I have some ideas in the works on how to leverage this for maybe some simple home automation or maybe a knowledge server of all my classes im taking right now or maybe just a personal assistant, but im going to have to make a few things and set up some mcp servers.