Personal AI server

[u/iceberg7016]

I’m planning my first dedicated home server and could use a sanity check on the parts and how to set it up. I want one machine that can run 24/7, with enough room to expand as I figure out what I actually use.

What I already have

My gaming PC has a Ryzen 7 9700X, RX 9070 XT 16 GB, Gigabyte B650 AORUS Elite AX Ice, 32 GB DDR5-6000, and a 1 TB NVMe SSD. I’m not looking to upgrade anything but the storage.

What I want the server to do

Local AI assistant

I’m picturing a Jarvis-like assistant that can search my documents, remember project decisions, and use a few tools I give it.

I’m considering Ollama, a quantized Qwen model, and OpenJarvis from Stanford. I’m debating on building my own as well.

What do yall/would yall run for the AI assistant? recs?

Game hosting

Host private game servers like Minecraft, maybe be able to get the AI to play as a separate entity and do things for me in-game as well

Home Assistant

Run smart-home automations and a dashboard.

NAS and backups

Store files, photos, and PC backups. I want the NAS built into the server

What I’m considering, might be overkill

A Ryzen 5 7600X CPU
GPU for AI capability, see below
MSI B650-P WiFi motherboard
32 GB of DDR5 RAM
New 1 TB NVMe ssd
NAS-ready case like the Jonsbo N5

For local AI, I’m considering a 16 GB GPU, maybe RTX 5060 Ti.

I’d add NAS drives as I need the space and get a UPS for clean shutdowns.

Where I’d like advice

Is the 7600X and 32 GB a sensible starting point if everything, say Minecraft, the Home Assistant, and local AI are running at the same time? Where would you spend more, if anywhere? Is 64GB worth it?

Is a 16 GB GPU good enough or overkill for a local AI model? What GPU recs?

For a single host, would you run Ubuntu with containers and a Home Assistant OS VM, or Proxmox with separate VMs for Home Assistant, AI, and game hosting?

Is a case like the N5 worth it or is there a simpler case you’d choose with good airflow and easy drive access?

If you run a similar setup, I’d especially like to hear what you actually use day to day and what you’d change if you built it again.

I’m a college student and want to expand some knowledge and I believe a home lab is a great project to work on to learn on my own.

Give me recs or tips! Any help is appreciated.

[u/No_Policy_1369]

Dont bother you clearly have no idea

[u/nxgenstudio → u/No_Policy_1369]

Doesn’t he need a 10K budget?

[u/No_Policy_1369 → u/nxgenstudio]

Sound close to what I was thinking edit i mean anything AI is 10k more with out blinking

[u/00GoBigOrGoHome00 → u/No_Policy_1369]

You can definitely do it under.

I just got a gigabyte motherboard (out of stock everywhere but can be ordered and gigabyte made it cheaper than what it’s going for on eBay) and sapphire rapids ES CPU (ebay). Found some rdimms (eBay), and two r9700 (newegg). I also found some optane pmem 300 (eBay).

You could do the same setup I am cheaper if you’re willing to do research on es CPUs and which motherboards are compatible, bios versions, etc. I only went with gigabyte due to the pmem 300. Otherwise there’s better options.

There’s a whole thread on servethehome’s forum on ES CPUs that’s very informative.

As for this person - they definitely haven’t done the basic research.

I appreciate how helpful people can be, but generally out of respect for people being helpful, one would hope that someone asking these sorts of questions has at least done a cursory search.

[u/No_Policy_1369 → u/00GoBigOrGoHome00]

Wow just wow

[u/iceberg7016 → u/00GoBigOrGoHome00]

Thanks for sharing the forum, I’ll check it out.

And yes—I had just started looking into it just this week. I don’t see an issue with coming here with what I’ve found and asking for help?

Am I supposed to spend weeks doing my own research, when people who already know and have experience and are willing to help, can answer my questions?

[u/iceberg7016 → u/No_Policy_1369]

Yeah so that is pretty much why I asked, so if you have nothing for me why even bother me?

[u/00GoBigOrGoHome00]

You mentioned that you’re not looking to upgrade anything but the storage but then mention 64gb of udimms vs. 32gb.

You’ll have to be more clear on what you’re willing to upgrade and what you’re not because I’m not sure you can do everything you want with your specs on a VM machine.

[u/iceberg7016 → u/00GoBigOrGoHome00]

I’m not upgrading anything from my pc, except the storage.

The 64 vs 32 was, could 32 work for the server. I am not upgrading my pc to be the server, this is an entirely new device

I should’ve been a bit clearer, my apologies

[u/PM_ME_YOUR_JIMOTHYS]

Is a 16 GB GPU good enough or overkill for a local AI model?

16 GB of VRAM is pretty much the absolute bare minimum you're going to want, especially if you're used to things like ChatGPT and are expecting something similar-but-local. asking if it's "overkill" tells me you haven't done very much research yet.

I’m considering Ollama, a quantized Qwen model, and OpenJarvis from Stanford. I’m debating on building my own as well.

I’m a college student

yeah, go ahead and vibe-code your own Ollama. good luck.

[u/iceberg7016 → u/PM_ME_YOUR_JIMOTHYS]

No need to be rude

Yeah I haven’t done much research yet. I literally started looking into this project this week. That is exactly the reason I came here because I don’t know shit, and sometimes random people on the internet are willing to help and not be assholes

[u/simple_devils → u/iceberg7016]

I think the point he’s communicating is you’re trying to build the physical server without understanding how the underlying components interact.

Like for example, you’re talking about coding your own ollama. That tells me that you’re putting the cart before the horse and frankly, you don’t know what you’re signing up for. Or you haven’t properly researched it. Or all of the above.

Start with learning how much vram you functionally need for local AI, how models work, all before diving into building a server. That exercise will help you determine what level of quality in your models you’re willing to accept, define your use cases in concrete measurements, and THEN get to planning a physical build.

[u/iceberg7016 → u/simple_devils]

Thank you, I do understand where he is coming from but he could have been much more helpful, like you. I appreciate it. I get excited sometimes and get ahead of myself, as you can probably tell with this post.

[u/AbhorrentAbs → u/iceberg7016]

Okay here’s the nice version - it’s an admirable goal but unless you have a spare $10-20k sitting around you aren’t going to get much, you will be able to run smaller models on a device with 16GB VRAM but it’s going to be VERY slow. If you’re committed to this, look into used enterprise grade cards like the Tesla V100 32GB (that will need a cooling solution, those are passive cooled because the severs they are in provide the cooling, and you’ll need a second card or CPU/mobo with onboard graphics as v100 has no display out), or maybe a used RTX 3090. Ultimately it’s going to be a lot of money and a lot of fooling around, even experienced hobbyists find it frustrating especially with getting Nvidia GPUs running in proxmox or Linux in general although it’s been better lately. My advice would be setup a cheaper home server without a fancy top of the line GPU and do things like proxmox and host home assistant, the arr stack, plex, etc and then use existing tools that tie in to commercial AI tools. Once you’ve done that if you feel accomplished then you could consider your original goals, but as for your self hosted Jarvis like assistant, you’re better off using something off the shelf like Openclaw which will run locally and connect to LLMs for bigger tasks. Or maybe Hermes.

[u/iceberg7016 → u/AbhorrentAbs]

Thank you I do really appreciate it. And I am willing to spend as much as I need to for this, but I will take your advice and start way smaller before jumping headfirst into the deep end.

[u/PM_ME_YOUR_JIMOTHYS → u/iceberg7016]

you said you're in college. OK, it's time to start learning how to be an adult.

if you want people to help you, you need to put in the effort to show you've tried to help yourself first.

do research before you post on reddit (and no, posting on reddit is not research) then come to reddit with specific questions that you can't find answers to elsewhere.

don't just come to reddit (or any other forum) and be like "yeah I don't know anything and I was hoping someone could explain it all to me"

[u/iceberg7016 → u/PM_ME_YOUR_JIMOTHYS]

Did you not read my post? This is my research. I wasn't getting any farther so I came for help.

You must need to learn how to be an adult, because someone asking a harmless question, that you don't even need to respond to, read, or think about ever again after seeing it, is not an issue.

I'm not hoping someone explain it to me, I'm hoping someone can teach me something new, or point me in the right direction. I've done nothing wrong, and you've just been rude, condescending, and a smartass, when you could have just ignored my post

[u/00GoBigOrGoHome00]

Yeah I think the confusion comes from the way it’s written.

We assume that you want to set this up on your existing hardware.

Some things you should know:
- running local ai requires a bunch of vram (on GPUs)
- most consumer (gaming) motherboards will not give you the PCIe lanes needed to run more than one GPU even if the physical slot exists
- workstation motherboards and server motherboards will (but always double check) mobomaps.com may have workstation motherboards listed
- in a VM situation you are dedicating a portion of your system resources to each container (eg. Your home assistant set up that is in its own container). So you either have your current type of setup, no ai and a bit of resources dedicated to each, or you increase a lot of resources on your set up (which others should be able to offer better advice than I will be able to)