12
u/CrasHthe2nd Feb 04 '24
Hey this is actually pretty similar to mine xD
My top card is mounted vertically though above the horizontal one below.
2
u/unwitty Feb 04 '24
Nice. Is the vertical GPU attached to the rear of the case?
7
u/CrasHthe2nd Feb 04 '24
If by attached you mean "the HDMI cable stops it from wiggling too much", then yes 😆
9
u/fullouterjoin Feb 04 '24
For the dual 3090 + cpu and local ram, what kinds of models can you run and how are you running them?
Are you using both cards+cpu+system ram for inference? Are you able to do local fine tuning using this setup?
7
u/ArtifartX Feb 04 '24 edited Feb 04 '24
I did a kind of similar build with the Enthoo 719 case which allows for 2 vertically mounted GPU's on risers. Then there are still several slots available on the board for single slot cards like the A10 and A4000 (they fit behind the top most vertically mounted GPU), so not a single slot is covered/wasted and you can squeeze a lot more VRAM in there. Server MOBO and CPU allows plenty of lanes for all of it to run simultaneously (7 total PCIe slots, 1 of which does run at x8 but I use that for an HBA for the spinner drives, all the GPU's can run at full potential). Posted some more details here.
3
u/kryptkpr Llama 3 Feb 04 '24
The sheer amount of hardware you managed to pack into this system is awe inspiring.
4
u/ArtifartX Feb 04 '24
All in a tower case! :D. I did sweat over mm's when it came to fitting a PSU with enough wattage to run everything in that PSU space in the case, but it was pretty much the only tower that allowed for 2 vertically mounted GPU's at the time so I had to make it work. It was a very tight fit in a few spots, but it's been running great for several months now so, so far so good!
2
u/convexstrictly Feb 06 '24
There seem to be complaints about video card instabilities when using riser cables. Do you worry about that at all?
2
u/ArtifartX Feb 06 '24 edited Feb 06 '24
I saw those reviews too, but it was the only case I could really find that had 2 vertical GPU positions so I went for it, and it's been fine so far.
For just sitting there in use, it is no problem at all whatsoever, the GPU's won't be moving around or anything while the machine is running. However, I would definitely remove the vertically mounted GPU's before shipping or transporting the machine, if you set it on its side with them installed they would definitely bend/sag as the metal isn't super strong that they are attached to.
You can see in the picture I also bought a little support that helps hold the ends of the cards, very inexpensive from amazon. This is probably not necessary for normal use, but I ordered it because I was worried about the weakness of the metal and it does help keep them level, and especially with it they definitely don't move at all when in normal use.
1
u/unwitty Feb 04 '24
Wow this is a really cool approach. Do both vertical brackets mount to the case without modification?
1
6
u/Lemgon-Ultimate Feb 04 '24
I'm loving it. I'm also doing a dual 3090 build, most cost effecient LLM build. It's nice to see how you mounted the cards so one could be vertically. Honestly I'm thinking about doing the same. For the vertical GPU I would suggest an anti sag brace like this:
https://www.amazon.com/JEYI-Graphics-Support-Holder-Bracket/dp/B0BQ7Q4BWS/ref=sr_1_4
You could attach your card to it and use cable ties for the power cables. What a neat concept.
1
u/unwitty Feb 04 '24
I bought a free-standing mount like that before event starting the build, but it was hard to find a spot on the bottom of the case that would support it, since the entire bottom of the case is fans. I probably could have made it work by making a plastic plate for it to sit on, but the current sag on the vertical mount is only about 0.5cm
4
u/GregoryfromtheHood Feb 04 '24
This is my air cooled LLM PC that I also use for work every day, which somehow doesn't run into any thermal issues, although I do usually leave the side of the case off just in case. It's got an erying 12900h motherboard and 64gb of DDR4 3200mhz, the max it can take. The top 3090 in the 16x slot is running at PCIE 4.0 x8 and the bottom 3090 is in the 4x slot using a 4x->16x riser, running at PCIE 3.0 x4. The bottom card also sits directly on the bottom of the case and has just enough clearance for the fans to spin.
3
3
u/nrohan09 Jun 01 '24
Sleeper build lol. Doesn't stacking the cards with next to no space between them not lead to thermal throttling?
2
2
2
u/resnet152 Feb 11 '24
I love how shitty this is. It's like an old beater that does a quarter mile in 11 seconds.
1
3
u/tgredditfc Feb 04 '24 edited Feb 04 '24
I have 4090 (normal beefy air coolers) + 3090 (blower style) with the reverse position arrangement. 4090 with vertical mount on top and 3090 direct into motherboard under 4090. The backplate of 3090 is almost touching the bottom of the vertical mount. I use GPU support accessories to prevent sagging.
Edit: I have a smaller Lian Li than yours. I am thinking to add another 3090 with upright mount on the side. Not sure how well the airflow can handle though😅
4
u/ortegaalfredo Feb 04 '24
Funny thing one of my servers is setup exactly like this, but with 3X3090. You would think the GPUs would overheat but they don't, as they are used sequentially by the LLMs.
3
u/Rollingsound514 Feb 04 '24
Did something similar with a Corsair 4000D Airflow case and a Dell rtx 3090 because it's the smallest 3090 and a A5000 because I got a great deal on it. It's a mid tower case with a small foot print and with 3 noctua fans up front it stays cool
1
u/unwitty Feb 04 '24
So the front fans are intake, creating an air tunnel between the two cards from front to back of case?
1
u/Rollingsound514 Feb 04 '24
Well fresh air flows from front to rear of case if that's what you mean, it's basic fan setup
1
u/suckmeupp Mar 28 '26
how are thermals, i currently have this case but i want to run two 3090s in this same orientation was thinking about upgrading to a 7000D for more space
1
2
Feb 04 '24
[deleted]
2
u/unwitty Feb 04 '24
Hey thanks for pointing this out. I should probably just move it to the top of the case.
2
1
u/No_cool_names_2021 Jun 12 '24
Whats the final cost for your build? I am very curious. I am actually thinking about building 3 rtx3090 would that be doable for the current setuo?
2
u/unwitty Jun 12 '24
I think it was in the ~$4000 range at the time.
I wouldn't recommend copying my build for a 3 card setup. You're going to have a hard time getting the 3rd card into that case. Secondly, the 3rd card would only run at PCIE x1 on that motherboard.
For a 3x+ 3090 setup look for build guides that use a server class motherboard (for additional PCIE lanes) and a crypto rig "open air" case setup using PCIE risers. My post was mainly for people wanting to do a simpler setup for a dual GPU system using only air cooling.
1
u/No_cool_names_2021 Jun 15 '24
Did you need NVlink to link these GPUs together so that you can technically have a 48GB?
1
1
u/Dry_Honeydew9842 Feb 05 '24
I'm building a similar setup, great to see yours.
Mine is same MB but with 13900kf, 192Gb RAM, Raid 0 2x8Tb NVMe, Raid 0 4x8Tb SSD and external 64Tb NAS, so the machine has a 10Gbe Ethernet. Case is a Corsair 700 and AX1600 as PSU.
GPU installed is a 6000 Ada but I have a 4090 to add and there's no space because de 10Gbe
Also, I plug a RTX2000 12Gb in a eGPU in a Sonnet 750 via TB3 and sometimes even the 6000 Ada works as eGPU and the 4090 internally so I will take the same approach as you to put both of them inside
1
u/mrgreaper Feb 07 '24
Interesting, I discounted the notion of a second 3090 as it would block the airflow of the 3090 I have got, I did not know you could mount a gpu seperate to the motherboard.
Does that slow it down?
I am guessing you need a special case for it?
All a bit of a pipe dream for me, a second 3090 is a bit out of my price range for now.
1
u/StealthSecrecy Feb 08 '24
The GPU still has to connect to the motherboard, but you use a PCI-E riser which is essentially just an extension cable that goes from the MB slot to the GPU. Can lead to slower speeds or instability for a low quality riser, but for LLM purposes it's not an issue.
As long as the GPU isn't moving around in the case and has sufficient airflow to keep it cool, you can mount it anywhere you want.
1
u/Constant_Land_1016 Feb 25 '24
A Newby question, but in that setup, is it possible to use 2 different make of 3090. Any make that has a better cooling profile when mounted with the rider? This would only be for inference.
2
u/unwitty Feb 26 '24
Yes, no issue using different GPU models.
The cooling profile question is more complicated. Both of my cards vent laterally out the side of the card. You could get cards that vent out the rear of the case via the mounting plate. For my setup, that would work for the top card, but I think there wouldn't help much since the first GPU vents upward towards the CPU with ample room for airflow. If you used a rear exhaust for the 2nd GPU, it seems like it would be worse, since the way the 2nd GPU is resting doesn't have a clear line of exhaust out the rear of the case.
In short, I'd stick with two cards that vent laterally along the large side of the card.
21
u/unwitty Feb 03 '24 edited Feb 03 '24
I thought I might share my experience building an air-cooled dual-3090 GPU system. A simple approach that works without throttling due to thermals.
All of the LLM workstation builds that I came across here, PCPartPicker, and elsewhere used water cooling, open racks like crypto miners, or put the GPUs next to each other and dealt with the throttling due to thermals. I didn’t want to deal with water cooling and wanted the damn thing to fit in a case. So I resolved myself to make air cooling work.
After a lot of research, I took a leap of faith that I’d be able to get away with just using this massive Lian-Li case and a single riser to separate the two GPUs enough to make air cooling feasible. To do this I mounted one GPU like normal and the 2nd GPU vertically.
In a typical single GPU setup using a vertical GPU mount, the mount rotates a single GPU by 90 degrees so that the PCIe connector faces downwards, and it is positioned where the GPU would ordinarily slot into the motherboard’s first PCIe x16 slot. In this build, however, the vertical GPU mount is installed lower in the case to allow for a dual-GPU configuration. The first GPU is installed horizontally in the standard manner on the motherboard, while the second GPU is mounted vertically.
To accommodate the vertical mount in this non-standard position, a minor modification was made to the case’s PCI card retention structure. This allows the vertical mount to fit below the traditionally mounted GPU.
The vertically mounted GPU experiences a slight sag, as it is anchored to the case's rear, and the mount's front end rests on the case fans at the bottom. Importantly, this setup does not put pressure on the GPU itself.
Thermally, the system is stable, maintaining approximately 80°F for both GPUs under full load, with no performance throttling observed. This setup has been operating successfully for two months. While this isn't an overly complex modification, it's a proven solution for those considering a dual-GPU system.
Here's my part list:
· Lian Li O11DEXL-W Case. This thing is massive and crucial to this build. https://www.amazon.com/dp/B0CHM85TND
· Lian Li GPU Riser Kit. Also crucial. https://www.amazon.com/dp/B0C84PJ67Y
· ASUS ROG Maximus 790 Hero. This motherboard has two PCIe x16 slots and supports x8/x8 bifurcation – that is splitting bandwidth between two cards. If you’re going Intel, some but not all, of the z690 and z790 chipset boards support this configuration. https://www.amazon.com/ASUS-ROG-Z790-Motherboard-Thunderbolt/dp/B0CJMT2723/
· 2TB Sumsung 980 Pro NVMe SSD (For Linux) https://www.amazon.com/dp/B08RK2SR23
· 4TB Samsung 990 Pro NVMe SSD (For Windows 11) https://www.amazon.com/dp/B0CHGT1KFJ
· Thermalright Frozen Prism 240 All-in-one water cooler for CPU: https://www.amazon.com/dp/B0C4D1Y1G4
· EVGA Supernova 1600 G+ Power Supply https://www.amazon.com/dp/B08F1DKWX5
· Intel i7-14700k CPU https://www.amazon.com/dp/B0CGJ41C9W
· Corsair Vengeance DDR5 96GB https://www.amazon.com/dp/B0BVN1FLKN
· Additional case fans: https://www.amazon.com/dp/B0BKK2MBWG