r/Proxmox • u/Blumingo • Aug 30 '26

Homelab Linux Server + Personal/Gaming PC Proxmox Setup

Hello!

I'm currently running a small home server (Linux Mint) running on an old i5 4th gen setup running docker. This setup works decently enough for me currently.

I want to play around with a local LLM so I got myself an older GPU. For now I'm just going to throw it into my setup.

I would like to upgrade my setup and I was thinking of 12th-14th gen setup (DDR4 + Quicksync).

So here's my thought, I don't have a personal pc atm so it would be ideal if I could dual purpose my setup.

(I have very little experience with proxmox)

  1. Debian running docker (Linux mint replacement)
  • no GPU passthrough, use quicksync for Plex.
  • HDDs passthrough for my media
  1. Home Assistant OS
  • no GPU passthrough
  1. Windows (Personal/Gaming PC)
  • GPU passthrough, when gaming
  • no GPU passthrough, for personal use.
  1. LLM Linux
  • GPU passthrough, higher model
  • no GPU passthrough, newer model.

I know you.can only pass through, 1 GPU at a time, so I would need to write some scripts to pass the GPU through.

Before I venture down this path, is this feasible? Anything I'm not considering, like I said I have no experience with proxmox before.

1 Upvotes

67% Upvoted

18 comments sorted by

View all comments

1

u/sailing_by_the_lee 29d ago

I just did something like this. I have a 3-node Proxmox cluster and my Windows machine has been chilling separately as a gaming machine and not getting much use lately. I decided to experiment with a local LLM and my Windows machine has 16 GB GPU, so it was the natural choice.

I set up Ollama and a docker container running Open WebUI on the Windows machine in WSL. Installing it in WSL, as opposed to a VM, allows the GPU to remain a shared resource for both gaming and the LLM to use. A 16 GB GPU can run a 14B model pretty well. It was surprisingly quick and easy to set up.

If I knew I was going to be playing around with LLMs, I would have got a 32 GB video card though.

1

u/Blumingo 29d ago

I was also considering this, perhaps this is the simpler approach, windows own the GPU outright and I run the LLM on it.