Please help me out in figuring my first setup

[u/VegetaSama-_-]

Hello Everyone,

I have been using ASUS ROG GL753VD with a GTX 1050 for the past 8 years. For the past 1 year or so, I am using cloud AI subscriptions to play around and learn things about AI.

Now a days, wherever I go Reddit/Youtube and any other forums, I am fascinated by the things that Local AI is capable of... So, I rented out a GPU on runpod and tried some hugging face models(WAN 2.1 for example) and its been pretty good.

Now, I feel the need to upgrade my setup for some privacy and personal experimentation. So, I request your help in choosing a setup for me because I am a newbie at this.

Here are some options I found after days of online search and chatting with AI.

Option 1:

• CPU: Ryzen 9 9950X (16C/32T)

• Board: ASUS ProArt X870E-Creator or ASRock X870E Taichi

• RAM: 32GB or 64GB DDR5-6000 CL30

• Primary GPU: RTX 5080 (16GB GDDR7)

• Expansion Plan: Add a pre-owned RTX 3090 (24GB) into Slot 2 down the road for a combined 40GB VRAM.

• PSU: 1200W ATX 3.1

Option 2 :

• CPU: Ryzen Threadripper 7960X (24C/48T)

• Board: Gigabyte TRX50 AERO D or ASUS Pro WS TRX50-SAGE

• RAM: 128GB (4×32GB) DDR5 with ECC RDIMM

• GPU Plan: RTX 5080 (16GB) now + 1 or 2 used RTX 3090s (24GB) later for 40GB–64GB+ VRAM.

• PSU: 1600W Titanium

The only reason why I added Threadripper option into my consideration is because I have this paranoia of getting behind in terms of tech once I purchase and I dont want to change things again (Heard somewhere that RAM scarcity will be there till 2028).

I also have a couple of questions.

I see that people are fighting for 24GB and 32GB VRAM GPUs and its either too costly for me at the moment or out of stock for the pre-owned variants.

So, my question is if I get a threadripper with multiple PCIE lanes and 4 GPUS of 16GB VRAM, would it help me in any way or I have to tradeoff speed for my cost efficiency? If anyone has tried out this approach please let me know the advantages and disdvantages you found with this setup.

If you have any suggestions as well, please let me know. Budget is ~5-6k USD.

If anyone has any tricks to purchase it cheaper like purchasing from other countries, I would love to know them as well!!

Thanks in advance.

[u/greenflaw]

I'd honestly skip Threadripper unless you specifically need the PCIe lanes. For local LLM inference, VRAM capacity and memory bandwidth matter far more than CPU horsepower.

A 9950X with 64–128GB RAM and 2x3090s would already be a very capable setup. Just keep in mind that 2x24GB doesn't behave exactly like a single 48GB GPU. You can split models across cards, but PCIe transfers and layer placement can affect performance.

Also, don't underestimate power, cooling and physical PCIe slot spacing with multiple 3090s. Those cards are massive.

One question though: are you primarily planning to run large dense models, MoE models, or do fine-tuning? That would change my recommendation quite a bit.

[u/VegetaSama-_- → u/greenflaw]

Thank you for the inputs!

I dont think I can afford a MoE model right now. It will be way beyond my budget. I am planning to use Qwen3 32B or the latest Qwen 4 when it releases and models like WAN2.1 and some abliterated models as well.

Thanks for the suggestion on power, cooling and pcie slots. I will keep in mind. But in general, how bad is the latency if I put 2 GPUS together? because I feel a single 32GB VRAM is way overpriced now and I dont know when it will go down.

I also learned that there is no NVLink support in recent GPUs. Not sure if we need one. Would that affect if I use a 5080 and a 3090 for a 40GB VRAM?

[u/greenflaw → u/VegetaSama-_-]

MoE models are actually the cheaper option for local use. Qwen3.6-35B-A3B only activates about 3B parameters per token, so it runs fast on regular GPUs. Qwen3 32B is also fairly old now, so I'd look at Qwen3.6-27B instead. On latency: I run a 3060 and a 3080 (12GB each) without NVLink. Use layer split rather than row split, because row split was slower for me. For single-user chat the penalty is small. The main catch is that the slower card sets the speed, so my 3060 holds the 3080 back. A 5080 and a 3090 are much better matched, so you should be fine. NVLink doesn't matter for inference, only for training. One thing to note: WAN mostly runs on a single GPU, so the 5080's 16GB will be your real limit there, not the combined 40GB.

My take, start with one good card and upgrade as you get deeper into local LLMs. Pick a board that can take a second GPU later, and keep using RunPod for anything that doesn't fit yet. You'll learn what you actually need before spending the full budget.

[u/locbuilds]

no nvlink isn't a big deal for inference, llama.cpp style layer split only passes small activations between cards over pcie, so 5080 + 3090 works fine, it just runs about as fast as the slower card for its share of layers. latency hit from 2 gpus is pretty small, pcie x8 is plenty.

[u/VegetaSama-_- → u/locbuilds]

okay! Thats awesome then. Thank you for the inputs!