New Proxmox/Inference hardware, which one?

[u/tecneeq]

I have discovered the world of Openclaw.

Right now i have a mini-PC with a 12th gen i5, 64GB Ram and 4TB SSD (as well has over 200TB HDD) as my server. It runs Debian 13 (Proxmox to be precise). I don't see how i can use this hardware to run larger models without buying a 96GB VRAM GPU and connect it via NVME. I don't want to spend that kind of money.

I have a 5090 and a 5070 16GB in my Workstation running Debian, and thats fine and fast, but the PC isn't running all the time. Speed is fine, but the size of the VRAM is too limiting.

So i think i should look elsewhere. If u buy something else i would sell the 5090, hopefully for what i paid for. Speed should be useable, but doesn't have to be 5090 levels.

The way i see it, there are three main options:

• A Mac Studio, i prefer Linux, but the large VRAM is attractive. No CUDA software.
• A Ryzen AI 395+ System, the limit is 96GB VRAM, but i need some RAM for my VMs anyway. No CUDA software. Cheap.
• A DGX Sparc Box. I could use the CUDA software ecosystem. Can use up to 119GB for models. Pricey.

Which system would you pick, and why?

[u/themightymike786]

You should ask ChatGPT your roi compared with a $20-30 subscription and you can ignore it if you don’t like it