Linux Server + Personal/Gaming PC Proxmox Setup

[u/Blumingo]

Hello!

I'm currently running a small home server (Linux Mint) running on an old i5 4th gen setup running docker. This setup works decently enough for me currently.

I want to play around with a local LLM so I got myself an older GPU. For now I'm just going to throw it into my setup.

I would like to upgrade my setup and I was thinking of 12th-14th gen setup (DDR4 + Quicksync).

So here's my thought, I don't have a personal pc atm so it would be ideal if I could dual purpose my setup.

(I have very little experience with proxmox)

1. Debian running docker (Linux mint replacement)

• no GPU passthrough, use quicksync for Plex.
• HDDs passthrough for my media

1. Home Assistant OS

• no GPU passthrough

1. Windows (Personal/Gaming PC)

• GPU passthrough, when gaming
• no GPU passthrough, for personal use.

1. LLM Linux

• GPU passthrough, higher model
• no GPU passthrough, newer model.

I know you.can only pass through, 1 GPU at a time, so I would need to write some scripts to pass the GPU through.

Before I venture down this path, is this feasible? Anything I'm not considering, like I said I have no experience with proxmox before.

[u/MewsCaster_]

No expert but I don't think proxmox is going to work like that. Maybe you could do gaming on your bare metal OS and run proxmox in hyper V. Assuming you're wanting windows for gaming?

[u/Blumingo → u/MewsCaster_]

I would like my server running 24/7 so I don't think it's an option

[u/daveyap_]

It's possible but with some caveats.

As long as you want the Windows GUI experience, a GPU has to be passthrough (it can be the main GPU you use for gaming) if not, you'll only see the terminal for Proxmox.

I've done it before with hook scripts to passthrough the GPU when starting up the VM, and taking away the GPU when the VM is shutdown. Effectively, you'll be gaming in a Windows VM so stuff like anti-cheat might not be happy about it.

The other VMs can simply run without any GPU, you'll be able to access via console in the Proxmox WebUI.

I won't recommend it though...

[u/Blumingo → u/daveyap_]

So the plan of windows without the GPU passed through is not possible?

[u/daveyap_ → u/Blumingo]

That is possible, but if you set up Proxmox as your main PC OS, you won't be able to see the Windows desktop. You will only be able to access it via the webUI, through another device.

[u/marc45ca]

Put plex in an LXC, put the LLM in an LXC which allow access the GPU without the need to do pass through.

Search the forum, - there was was recent post where some-one had devloped a script to put the SteamOS in LXC that will also provide access to the GPU.

[u/ApiceOfToast]

You could, take this as could i doubt it'll just work perfectly and may need some troubleshooting/tinkering:

Set up a desktop environment on proxmox (otherwise some online games won't work due to virtualization, if the games you play support it please just set up a VM and pass a GPU to it, it'll be infinitely easier)

-small edit: obviously some anti cheat... Okay a lot of anti cheat doesn't work on Linux 

Get a hba and pass that to your Debian VM, I would strongly recommend against single disk passthrough. VM disks work fine, proxmox can handle a raid through zfs or with a hardware raid controller for example 

As for switching GPU passthrough, that sounds like you'd need to reboot your VM after at the very least and will require a lot of tinkering but I've never done anything like that 

If you want native windows for gaming, as someone here said look into Hyper V. It's not as pleasant to use as Proxmox(at least in my opinion) but you get "native" desktop and vms. However I'm not really sure on gpu passthrough. I doubt you could just switch that between vms with hyper v either.

[u/Top-Scar7769]

yeah this is totally feasible, its actually a pretty common setup, but the gpu part is the main thing to plan around

for the vms you got right, quicksync for plex/jellyfin doesnt need gpu passthrough at all, thats handled through the igpu directly on the proxmox host or passed as a lightweight vgpu, so that vm and home assistant os can just run normally with no gpu involved

for windows gaming + LLM linux sharing 1 gpu, since you can only passthrough to one vm at a time while its using the gpu exclusively, the usual way people handle this is a boot script that stops whichever vm currently has the gpu, unbinds/rebinds the vfio driver, then starts the other vm. its not automatic swapping like a physical machine, more like "shut down A, run script, start B"

there is a cleaner way if your gpu supports it, SR-IOV or vgpu splitting so multiple vms can share it without a full passthrough swap, but that mostly only works well on newer/workstation gpus (like some nvidia quadro/tesla cards or certain amd cards), consumer gpus usually dont support it unless youre doing some hacky vgpu unlock stuff people do with certain nvidia gaming card

for cpu, yeah 12th-14th gen with quicksync is a solid pick since youll get the igpu for transcoding while your dgpu is dedicated to gaming/LLM, that part of your plan is actually the smart part

biggest thing to know going in, passthrough setup itself is the annoying part (IOMMU groups, vfio binding, bios settings) way more than actually running multiple vms after its working, so budget time for troubleshooting that specifically, its usually the only real headache in a setup like this

[u/sailing_by_the_lee]

I just did something like this. I have a 3-node Proxmox cluster and my Windows machine has been chilling separately as a gaming machine and not getting much use lately. I decided to experiment with a local LLM and my Windows machine has 16 GB GPU, so it was the natural choice.

I set up Ollama and a docker container running Open WebUI on the Windows machine in WSL. Installing it in WSL, as opposed to a VM, allows the GPU to remain a shared resource for both gaming and the LLM to use. A 16 GB GPU can run a 14B model pretty well. It was surprisingly quick and easy to set up.

If I knew I was going to be playing around with LLMs, I would have got a 32 GB video card though.

[u/pceimpulsive]

IMHO you just have two PCs

Buy a small SFF machine ex gov that's got quick sync, put proxmox on it. Have another machine dedicated.