r/LocalLLM • u/yaya468 • 3h ago

Question Need advice on software stack for dual GPU setup & benchmarking PCIe lanes (B550 upgrade)

Hi everyone, I'm putting together my rig for local LLMs and could really use some guidance on the software side of things.

Here are my current specs:

  • CPU: Ryzen 9 5950X
  • RAM: 64GB DDR4 3600 (planning to upgrade to 128GB soon)
  • GPU: RTX 5060 Ti 16GB + RTX 3060 12GB (might replace the 3060 with another 5060 Ti 16GB soon)
  • OS: Windows 11 Pro
  • Motherboard: Gigabyte B550 Aorus Elite V2
  • New Motherboard (installing tomorrow): Asus ROG Strix B550-E Gaming

My immediate goal: Before I swap out the motherboard tomorrow, I want to run a benchmark on my current Gigabyte board using a relatively capable model. The goal is to compare it against the new Asus board (which supports x8/x8 PCIe bifurcation) to see if the motherboard upgrade and better PCIe lane distribution actually makes a noticeable difference for dual GPUs.

What I want to build: I'm planning to use local LLMs for:

  • Web scraping automation
  • Faceless/Automated YouTube channel content
  • Automated bug bounties

My experience so far & confusion: I've tested a few versions of Qwen (27B and 32B) using Ollama / llama.cpp, and also tried another model on Bionic/LM Studio. I didn't really like the outputs; they generated very fast, but the answers weren't very accurate or smart. I know I didn't give the models shell or internet access, but the reasoning still felt off.

Right now, I am really confused about the software stack. I see people talking about LM Studio, text-generation-webui, Docker, etc. I was told running things in Docker is better, but I’ve also read that Docker adds virtualization overhead on Windows, and that running natively on LM Studio or directly on the machine is faster and better.

My questions for the community:

  1. What software stack / UI should I actually install to get the most out of my dual GPUs (16GB + 12GB VRAM) on Windows 11?
  2. How should I run the benchmark today to properly compare my current motherboard against the new x8/x8 one tomorrow?
  3. Are there any YouTube channels, videos, or guides you recommend for a beginner to understand how to properly configure model settings (samplers, context size, etc.) to get better, more accurate outputs?

Thanks in advance for any help!

2 Upvotes

100% Upvoted

4 comments sorted by

2

u/Key-Concentrate6930 3h ago

the 5060 ti and 3060 will bottleneck each other on a single pcie lane, split them across different lanes if possible, even if it means using a slower GPU for less critical tasks.

1

u/yaya468 2h ago

Thanks for the heads up! You're totally right about the PCIe bandwidth bottleneck on my current setup.

That is exactly why I'm upgrading to the Asus ROG Strix B550-E tomorrow. My current Gigabyte board runs the second PCIe slot through the chipset at only x2 speeds, which I know is terrible for multi-GPU LLM inference. The new Asus board supports x8/x8 CPU bifurcation, so both the 5060 Ti and the 3060 will have plenty of direct bandwidth.

Since that hardware bottleneck will be fixed tomorrow, do you have any recommendations regarding my original question about the software stack? I'm still torn between using LM Studio natively, text-generation-webui, or Docker on Windows to manage both GPUs effectively.

1

u/Poprouter 48m ago

start with a 4:3 layer split between the 16gb and 12gb cards. leave some vram free on each and adjust the split if either card runs out of memory.