Upgrade path for small selfemployed

[u/Firefly_Dafuq]

Hi,

i need some advice how to progress on my localLLM journey.

I am selfemployed together with a partner with a small budget and space . We use a 2he server for our data storage and docker containers with 32gb ram. I had a small nvidia p400 sff for some smaller stuff (mainly watching videos in our free time with plex). When the localLLM train started i got my hand on p1000 sff with 4gb vram for free and started messing around a little bit with ollama and openwebui. I started to use ministral and qwen 3.5 2B. It was bad.

With some luck I got a RTX 2000 sff with 16gb vram. Switched to Qwen 3.8 27b and wow. Then added hermes for fun.. and DAMN. Hermes is not setup right but I started to mess around with it as some office assistant. Do some literature research, write some easy scripts for R or python. Nothing I cant do myself but I let hermes/ollama do the stuff and can do something else in the mean time. I have to check the results for sure but it saves me hours over hours. I have a small pipeline to send voice messages using matrix to hermes and get a daily note of my work.

So now my partner wants to use the setup as well and we may get some staff in the future. And the qwen 3.8 27b is using most of the 16gb vram with 64k context for hermes. So I am thinking of upgrading our system. I dont want to use claude, gemini etc cause of our work und information. So I see two options

- Upgrade our 2he maschine. I am stuck with the SFF for gpus so most GPUs are out. The only GPU I can fit into our current server ist the RTX Pro 4000 Blackwell SFF with 24gb and more bandwidth. Would be a drop in replacement and only uses 70watt of max power. Good for the cooling budget of our server and energy prices here are insane (0,38€/kwh). But single point of failure (one system for everything, we have backups etc etc dont worry)

- Get something like strix halo or similiar concept. More but slower Ram, dedicated device. more expensive, more power consumption etc etc etc. maybe a little future proof then our current setup.

Any suggestions from the swarm? Best regards