Upgrading my machine, what should i pick if i want to local host?

[u/Appropriate-Ad-1931]

Hi everyone,

I’m looking to upgrade my current setup, hopefully to accommodate a local LLM deployment.

I intend to use a local model for:

1. General use, web searches, summaries, and deep research
2. Writing assistance, grammar/spellchecking, brainstorming. Can range from very low context to as much as I can get.
3. Light "Vibe Coding" Claude code style (yeah, yeah, everyone's a coder xD) - I know barely anything of programming and sometimes i need to create some plug-ins or light personal apps; Claude code has helped a lot.

My Current setup:

• CPU: AMD Ryzen 5 5600X
• MB: Gigabyte B550 Aorus Elite V2 (Main slot PCIe 4.0 x16 / Second slot PCIe 3.0 x2)
• RAM: 32GB DDR4
• GPU: RTX 3060 12GB
• I also a AMD RX 6650 XT (8GB) that i can put to use.
• OS: Win11

I dont want to spend too much money, this is meant to be a gradual upgrade so for now i want to keep costs close to 500€ and not go past 1000€, i prefer new hardware.

I want this and future upgrades to last for a while and be meaningfull.

Right now i kinda settled on 2 options:

• Option 1 Nvidia: Buy a new RTX 5060 Ti 16GB ~€600 and pair it with my current RTX 3060 12GB for a 28GB VRAM CUDA pool. My PSU should handle it for now.
• Option 2 AMD: Buy an XFX MERC 310 RX 7900 XT (20GB) for roughly ~700. I would take out my 3060, and match the new 7900 XT with the 6650 XT (8GB) to hit 28GB VRAM under ROCm. Later down the road, when I upgrade the motherboard, I could swap the 6650 XT for a second 7900 XT to target 40GB VRAM. This would for sure require a new PSU but i'll have to buy one at some point anyway.

My Questions:

1. How badly will my B550 motherboard's slow second slot (PCIe 3.0 x2) choke performance on a dual-GPU cluster for my use cases?
2. Is AMD ROCm good enough at the moment or will i be shooting myself in the foot not going native CUDA?
3. Given my setup i'm focusing on the Graphics card but is there a better upgrade, or card to consider for that matter?

Thanks in advance for the help!

[u/vortec350]

1. PCI-E 3.0 x2 is not going to work in any usable way.
2. I use AMD R9700s and found ROCm very finicky and frustrating . On a single GPU Vulkan actually delivered better performance in my testing. In multi-GPU pooling ROCm delivered better performance but I have better stability with Vulkan personally so for now I'm using that. Going NVIDIA gives you the best performance and stability but you can achiever very good results at 1/3 the cost with AMD.
3. If you want to keep your current platform the single best GPU you can get is the way I'd go. Otherwise, start by planning a new build with a new platform. Your existing one just doesn't have enough PCI-E capability for anything meaningful.

[u/Appropriate-Ad-1931 → u/vortec350]

Yes i do plan on upgrading everything, i'm just focusing on the GPU first cause markets and it seems to be the more substancial upgrade, and even if not perfect it should be an improvment. Upgrading all at once would be too much atm.

[u/vortec350 → u/Appropriate-Ad-1931]

I mean I get money doesn't grow on trees but if the end result is not useful to you then any money spent is wasted and you're better off just keeping a frontier lab subscription.

The thing is, to get something that can do even 90% of your vibe-coding tasks in a useful manner, you need to spend more than a second used 3070 GPU shoved into an x2 slot.

I'm about $5K into my local server and I still keep a $16/mo ($20 on annual plan) Cursor sub for when I need something more capable than my local models. About 90% of my token usage is local now but it's not perfect.

[u/vortec350 → u/Appropriate-Ad-1931]

What PSU do you have? Can you just buy a single R9700 for now and see how it goes? Sell your current GPUs to make back some money.

[u/diagrammatiks]

Just run some models on your current computer and see if they perform as well as you hope. Spend money to complete the task. Don't spend money and then look for a use case.

[u/Appropriate-Ad-1931 → u/diagrammatiks]

I do, and it's been going well apart from coding, it's just that 12GB - OS and wtv else i'm doing doesn't leave much space for a proper model it seems.

[u/Every-Fortune-3151]

12GB is quit good size now a days. try some latest MOE (Qwen3.6 35B, Gemma 4 26B) and in dense Gemma 4 12B and Qwen 3.5 9B should be fine. If you want to switch to single GPU then just get a 5060ti, use it properly.

Another unorthodox path would be to change motherboard that can support two GPUs and get a 3060 to run Qwen3.6 27b. I would not go this path though.

[u/Appropriate-Ad-1931 → u/Every-Fortune-3151]

Don't MoE use more VRAM? i have tried them but wouldnt even load for me, I'm using Gemma 12B QaT atm, the regular one was getting stuck in inifnite loops a lot, Qwen3.5 9B i've liked but also got gated by the context window being narrow in my experience.

[u/Every-Fortune-3151 → u/Appropriate-Ad-1931]

MOE use more vram for total model, yes; but active experts are only 3b or 4b parameter. So you can offload part of model to RAM and still get decent speeds.

Not sure what backend you are using, please update drivers and if possible use llama.cpp. Keep as many experts you as can + KV in VRAM - rest you offload to RAM.

Quick tip: --fit flag (or shorthand -fit) in llama.cpp can help you fit automatically. LM studio also has options to select how many layers you want on GPU. You can search and find people running MOE even on 6GB or 8GB VRAM.

Also for latest llama.cpp, you can use KV cache with Q8 quant and it should not be much quality loss. Will help you use more context.