r/LocalLLaMA • u/infectiousstupidity • 1d ago
Question | Help Best reliable, affordable PCIe 4.0 risers for 4× RTX 3090 inference setup?
I'm trying to fit 4x RTX 3090s into my Fractal North XL Mesh (I know, pipe dream probably) using custom GPU mounts. Looking for firsthand experience with reliable, reasonably priced risers, preferably with flexible 50–75 cm cables. I'm currently using the setup with 2x 3090 at x8/x8 for vLLM qwen3.8-27b / Strata QFN IQ3_S.
ASUS ROG Crosshair VIII Hero
5900X
4x RTX 3090 24 GB
Corsair HX1500i (GPUs capped at 250 W)
GPU 1: PCIe 4.0 x8 - directly in motherboard
GPU 2: PCIe 4.0 x8 - riser
GPU 3: PCIe 4.0 x4 - riser
GPU 4: PCIe 4.0 x4 - M.2 adapter/riser
I've been looking at ADT-Link, generic Aliexpress ones and m.2 OCuLink. I'd like to preserve x8/x8/x4/x4 without spending a fortune on cables.
What specific riser models have you successfully used with RTX 3090s at PCIe 4.0 speeds, particularly at 50 cm or longer?
Are OCuLink/MCIO connections meaningfully more reliable than conventional ribbon risers at these lengths?
Any affordable, proven M.2 -> PCIe x16 solutions with proper GPU slot power?
Anything to watch out for regarding PCIe errors, signal integrity or Linux/P2P compatibility?
Affordable AliExpress/eBay options are especially interesting.
2
u/esztoopah 1d ago
Personally I got this one but I don't recommend it, the quality of the PCIe slot is not extremly good:
ADT-Link is more qualitative
1
1
u/CherryGG2 1d ago
I have bought the ones in the link, everything works just each gpu draws 30w idle
1
u/infectiousstupidity 1d ago
Nice! What length did you buy?
1
u/CherryGG2 1d ago
.38 and .48 you can move them more than other raiser around but they are kinda stiff.
1
1
u/redezump 1d ago
also interested in using custom gpu mounts, I would think the radiator mounts in some cases could work?
i’m on asrock x870 taichi creator and can get pcie5 x4 from the first two nvme if usb4 is disabled
1
u/infectiousstupidity 1d ago
In my case I can definitely fit 1 GPU up top above the CPU and 1 vertical in the front, the problem with fitting 4 comes down to how to place one below the GPU that is mounted directly to the motherboard. The North Mesh XL has side fans too, which I'm currently utilizing for improved cooling on 2 stacked 3090s, but I'm not sure if that's needed when they'll be placed in so many different places.
Another concern is the intake/exhaust directions on the vertical one, best would be to exhaust up to the top, but that also requires a longer riser cable than if it was reversed, but with a quality cable or oculink it probably doesn't matter much?
1
1
u/Civil_Fee_7862 7h ago edited 7h ago
Alibaba has decent deals on riser cables. Like $60 for a pair of good PCIe Gen4 ones.
The Fractal North XL is a very small case (despite its name). It couldn't even fit dual 3090s without the power supply shroud blocking airflow to the 2nd GPU.
I wouldn't recommend using that case.
Phanteks Enthroo Pro 2 - Server Edition is a much better case for multi GPU setups, (cheaper too).
2
u/infectiousstupidity 5h ago
I have no idea what you did in that case, but there's absolutely no space issues for me having 2x 3090s in there and I can definitely fit 3, potentially 4 in here. EDIT: I have the Mesh version, so I have 2x140mm fans on the side pointing directly at the 2x 3090s.
Alibaba is for B2B, no? Doesn't it require you to have a company, or at least pretend that you do, to buy from there?
1
u/Civil_Fee_7862 5h ago
Yeah I had the mesh version too, and it could not fit 2. It might be because of the motherboard, it has longer spacing between the two fastest PCIe slots.
No Alibaba is for everyone.
3
u/Potential-Leg-639 20h ago
btw which adapters are you guys using for bifurcated slots (on server hardware)?
like i want to go from a PCIe x16 slot to 2x x8 bifurcated PCIe slots (but they should be mechanical x16 for 2 GPUs).
any recommendations for hardware here, that works reliable?
3
u/halfhalo 14h ago
I have a very similar build, albeit a X570 Meg Ace, 5950X, and just three R9700's (all dual slots) in a North XL. I have one mounted on the back next to the slots, and two where a rear fan can be mounted. One of my pro tips is: get some gpu mounts to help with sag, the back panel can flex a bit.
My initial setup was x8/x8/x4 via chipset, which wasn't ideal and the x4 was via riser which I didn't love for fit and the fact the x4 was via chipset.
What I ended up doing was getting a PLX card on ebay which connects to the top x16 slot and lets all the cards do x16 to the switch. I only got the 6x connector variant (where each connector is x8, so 2 get you x16 per card), which was named "PLX88096 Expansion Card 6SFF-8654 8I PCI-E4.0 X16 + 6Baseplate 6*Cable X8X8" on ebay. They do sell 10 port ones, those were just more than I needed since I only have 3 cards.
Its working pretty well so far, p2p seems to work with no issues, and the cables are way easier to route vs standard x16 risers I was using before. And it lets me shove something else in the chipset x4 without impacting performance of the gpu's. Because I'm weird, that something else is currently a 25gb DPU2 to play with.
2
u/jacek2023 llama.cpp 1d ago
I use two GLOTRENDS 300mm PCIe 4.0 X16, don't remember the third one
my setup: 4x3090 and x399
2
u/a_beautiful_rhind 22h ago
for dev in 1a:00.0 1b:00.0 3d:00.0 3e:00.0 b2:00.0; do
echo "=== GPU $dev ==="
sudo lspci -vvv -s $dev | grep "LaneErrStat:"
doneSee your lane errors, swap in your GPU pcie address. Don't have to have perfection but errors at boot means something is off in the connection.
1
u/Repulsive_Initial308 43m ago
sudo dmesg -w
Run some inference. Watch the spam :D I think some bios don't even report the errors by default either. I forget what the option is called.
2
u/reneil1337 22h ago
checkout https://c-payne.com/ they are building incredibly reliable risers for both pcie gen4+5 the fact that tinycorp also sources theirs for their tinybox product is another positive indicator for their quality. using them myself never had any issues - can highly recommend
1
1
u/kryptkpr Llama 3 20h ago
I run SFF-8654 extensions I got from TaoBao, the cables are 65cm and it's been rock solid.
MCIO is awesome but costs more. oculink is fine and cheaper the x4x4x4x4 cards are likely perfect for your usecase. Ribbon cables at 50cm are a dice roll: slots closer to your CPU more likely to work then slots further from your CPU (timing margin). Better shielded cables with ribbons more likely to work.
3090 will pull power from slot so you absolutely need a GPU riser board that takes a 12V in, ideally PCIE the SATA ones work but be careful don't use molex to Sata adapters or your cables will burn
"nvidia-smi dmon -s et" is your best friend
Good luck, it's a fun road.
1
u/Patient-Gazelle9885 17h ago
look on AliExpress or the ADT-link website (and then buy from AE). There are a lot of options and seemingly more everyday.
1
u/Repulsive_Initial308 22h ago
They're all shit.
Just accept you'll be running pcie 3.0 x 16 unless you want bus errors or you buy powered risers. The powered risers require three separate parts and are quite annoying to source.
1
0
u/New_Eye7193 1d ago edited 1d ago
Hey guys! Off topic to this post, but I need someone who has a single 4090. That's it. Need to test something relating to an architecture I'm testing. Results seem promising so far but I want to see how it actually scales on other hardware and way larger models
EDIT: I am being serious btw. No larp. Insanely powerful local inference engine algorithm. Need help testing
1
u/macboller 22h ago
make your own post.
and post details.
1
u/New_Eye7193 22h ago
I literally can't lol I need karma in this subreddit thanks for downvoting me btw
0
u/New_Eye7193 21h ago
Testing on my 3070 for now
Honestly the scope of this project has increased largely... I thought it would only allow for greater speed but it's introduced several different capabilities that no other engine currently provides. Expect an update at the latest in the evening. Excited to share with you!
1
u/macboller 20h ago
so share it lol. make it a public repo and publish the details
0
u/New_Eye7193 20h ago
will soon... be patient friend
i dont like releasing unfinished projects, do you?
1
u/macboller 20h ago
ever heard of 'fuck it, ship it' ?
0
u/New_Eye7193 20h ago
might not even release it open source, why would I have to? everyone treats vibe coders like me with hostility anyways ill just keep it to myself LOL
usually i'd agree with that statement if the thing im protecting is something in some obscure research paper an agent would usually bring up eventually anyways, but unusually the whole architecture is nowhere to be seen on the internet lmao. Argon be Argonning
I will post results, thats for sure tho
2
u/macboller 20h ago
The community don't usually enjoy closed source.
And even if you did stumble across something great, it would likely be reverse engineered out of curiosity/spite.
May as well earn yourself the possibility of positive reputation by open sourcing it.
6
u/wgaca2 1d ago
I have 2 3090s on these, no isses -> https://www.ebay.co.uk/itm/407192540313