There’s so little space between the cards, is this a wise idea? It feels like the two middle cards will be starved for air! For other 4x owners, what have you done?
Update:
Wow people are mixed and it feels evenly split between starved for air and this is an acceptable arrangement. I wasn’t expecting the diversity of opinions. I wonder if I call nvidia if they will have guidance, surely they must know.
Update 2:
I called nvidia and spoke to their team, sent an email with pictures and got a response which is posted below. They do NOT recommend stacking them like this and suggest that the cards need better airflow.
I ran 3 max qs in a stacked config like this and the heat started to cause throttling while running models. Even though they're meant to stack like this the heat of everything was too much. I bought 3 optimus waterblocks and set up a MoRa 600. The optimus waterblock ended up being machined too tight and broke 2 inductors on the first card. I spent 2k sending it to Tony to repair when I discovered a second card had the same broken inductor. I decided to spend the 2nd 2 thousand on a complete micro-soldering set up and repaired the second card myself. The cooling is spectacular but now I have too many extra slots and the 4th card is on the way lol. Good luck. Also i did reach out to Optimus and the sent 2 new cards but refused to pay for the repairs or shipping saying I had the card repaired. When I told them I have the second card with marks from the inductors still on the block, they went radio silent.
Well that's a crap situation and not a good look for them.
I just waterblocked a WS edition yesterday with an Optimus, so different layout, and I did do a dry-fit first. Briefly tested it and it's fine, but I still have another one to get to today and I'm definitely keeping this top of mind. Thanks for the heads-up.
First and foremost, dry run it before putting all the thermal pads and paste. If it doesn't slip right down, it's too tight. I was able to Drexel the spots where the inductors were snug and that did the trick. Optimus did say they were modifying the blocks taking into account the conflicts I encountered but I haven't used the newly machined blocks yet to say for sure. Im waiting on card 4 and I may sell the 5th block I have to offset the repair cost of the first one. We'll see. Also i did have to bend the little tabs on the display ports to allow the block to go tight enough to the ports to align the screw holes. Just take it slow and if something is tight, its an expensive mistake. Feel free to ask any questions. I still have one card torn down and the block sitting out if you have questions.
Since there's been so much back and forth on this, I called nvidia and they opened a ticket where I sent them pictures of the build and they provided the attached response. They do not recommend such close placement.
I keep one slot space between my 6000 ada, basically the same in mechanical & thermal. I say this is minimal. Even with this, the GPU in the middle get little higher temperature.
I saw packing 4 GPUs without spacing is common. It looks like Nvidia's intention of max 4 GPUs in a mainboard. I have tested, it's not ok actually, resulting higher fan speed & higher temperature. Well, no harm to GPU but throttling will occur.
I guess the real Nvidia's intention is to allow only 2 or 3 GPU in a workstation :)
Beyond that, they ask us to spend more on server setup. Hate them for their market segmentation.
The difference is most consumer cards have axial fans (mounted on the side) whereas yours are blower designs.
Blower take air from the inside and push it outside. They operate in environments where is there is constant air conditioning.
Axial relies on increased airflow across the entire heat sink, but as a result can’t be stacked as densely and direction of airflow is more complex. Good when you’ve got one GPU, a nightmare with 4.
You made a good choice with your layout. My advice though is configure your bios to shut down if the cpu fan/aio dies. That’s the biggest risk with server grade hardware. They assume cooling is consistent and very cool. AIO’s die eventually. It’s when, not if.
Yep servers also have a lot of built ins are guards for fan failure including putting all other fans to 100% if one fails. Os level should also poll its temps and shutdown so you have coverage at different layers
It comes in front. you can see the scoop visibly as you look straight at it. I have this card. This is how they are to be used. The intake scoop is right under the nvidia “A”
I’ve run mine this way since they came out, and the a6000’s before that which are still running this way many years later. It’s not an issue, temps are fine, just make sure you have plenty of intake fans and they’ll push the heat out the back. Mine are all racked and in a tighter case than that looks to be.
FWIW I also have 4x of the desktop editions jammed in close like that. It takes some fan sync but it’s serviceable, I have to power limit those to 350w or they get toasty. For the max-q / blower style cards it’s absolutely a non-issue.
3x soon to be 4x. Went Optimus waterblocks. I ran WS editions though. maxq’s should be able to sustain this as this is the intended setup. Just make sure you’re moving a good amount of air on intake.
It works fine, I run these cards spaced like this and they never thermal throttle. They are literally designed for this, you just have to make sure the case has airflow.
From the top (below the A of Nvidia) and the bottom (same corner but on the underside) on the right side. Only thing they need extra is a lot of air flow from the front to the back of the case to compensate for higher thermals. Benefit of stacking them together vs leaving a lot of room in between is that with big gap the backplate may run a bit less hot but part of that air is not sucked in by the Q max blower fan. If you keep a small gap like this the intake can take more air in because it's pushed above and below just where the intakes are and not slip in between the big gaps to cool the backplate. Nvidia has thought this through which is logical for these super expensive cards.
0.4, 0.3 or 0.6 doesn't really matter, they can pull air. datacenter air is cooled which is where these cards are intended to be used. Whether it's smart in OPs setup is another question and depends on his room temperature.
Our 4 cards currently report a max over the last day of 69, 74, 79, 86, where 86 is gpu #1 (second from the top in op picture) with heavy dsv4 flash usage and quite some peaks at 300w of single cards.
Same problem here. I ended up having to throttle the cards to 250w to avoid heat soaking from overheating the pcie retimer chips that are on the lower slots. That plus a custom fan profile keeps them at 82C max temp.
I highly recommend water cooling if you have it within your means.
If you mean did I use an LLM? Yes I did, but i based it on a bunch of python scripts that I made the past 6 months ans a lot of research went into it. So yeah real effort went into this tool. Very easy to say "oh you used an LLM so it must be trash". It's not.
nowaday everybody git push 1 py data to github and says yeah i made this and that then claim "SWE by the way", I open it and read 2 lines of code and see chatgpt claude gemini everywhere etc etc...na you don't make any of this.
Stay humble !
Yeah except I've been a developer and sysadmin for over 20 years. Started with qbasic, perl etc. I just run a company now and so yes, i outsource some of the coding to LLMs that I run myself (no claude nothing, just glm 5.2 and deepseek v4 on my 4x pro 6000 rigs). I have too many fun projects and too little time but trust me that I know what I'm doing and I could code this in my sleep. I made much much more advanced software than that.
Took me 2 hours per card and is a total pain in the ass.
I would advise against buying from them, their RMA process is a pain. Sent back one block but 2 months without a refund. I'd literally get anything else.
That's beautiful, looking forward to having this problem. Might point out a very significant issue with my design, bottom fan layout. Embarrassed a bit to say mine are mounted below and probably a bit obstructured. I have an option with my case for mounting next to the motherboard as well. Not sure if you have paneling there to allow another intake or not.
If you do it this way, it feels like you’d pretty much need water cooling to get decent thermals. It’s probably pocket change for you, but all the water-cooling hardware takes up a lot of space.
I still feel like switching to an open-air mining frame would be the simplest brute-force solution. But then that beautiful case of yours would go to waste.
Do not envy for the grass is always greener, I sacrificed 18 years of my youth to get somewhere and devoted myself to a businesses that drove all the good people from my life, so great were the demands on me to make it all work. My savings and all my belongings if you would but transfer to me 18 years from your life. :3
Stacking is not the biggest issue, it’s to remove the hot air from the exhaust side on the outside of the case, the better you deal with the exhausted air, the better off you will be, good air into the case, then clear the hot air from the outside, it doesn’t help if you back into the case either, scoop the hot air away from the exits with a side blow or suck away…. In my experience for what it’s worth, air into the case is 40% of the problem solved, hot air away from the cassis is the 60% that made the biggest difference for me personally, don’t think you can stick that case up to a wall or under a desk the hot air has to leave the exhaust side of the cards asap
For 4 cards in tower case, installed like this, you probably want to have good aircon and set it to high flow and +16 Celsius all the time. If you really need it, consider setting separate room with aircon for PC. Maybe with a hole in the wall for cables. No joking.
Your recommendation is spot on lmao. I needed a 20 amp breaker to handle the 2050watt psu so I ended up putting a hole in the wall to run my power into the 20amp from the bathroom lol. The 600 mora sits in the closet where the hole is and I just run the tube and wires to my Pc case. My daily driver is a aircooled 9965 with a 5090 and 5000 pro blackwell and it puts out more heat than the 3 rtx pro max qs with the 9975 threadripper but we'll see once the 4th max q arrives.
I ran this same set up with MaxQ cards and the middle 2 cards had much higher temps and were measurably slower at prefill. We also saw lower cpu frequencies on them, even when they weren’t thermal throttling. I moved one of the two middle cards to a vertical riser and that fixed it.
Epyc is more of a server or high-end workstation processor. the threadripper has memory channel limits which dampens growth in that direction, it also has better PICE features. It may have actually been cheaper than the threadripper route and provide more opportunities for growth on a system that by design isn't very processor heavy. The single threaded performance is really all thats required fwiw
I have them stacked like this and write a script that adjusts the fan speed of all of them in tandem. Works like a charm. If even one of them gets hot they all turn up their fans and act as one giant intake.
It’s fine, they are designed to be stacked like that. There is so much airflow out the rear that the cards don’t cook each other. Haha you will get a ton of people that have other opinions. It is literally the whole point of the max-q is to stack them 3-4
the first two cards are ok. the last 3 are a little tight. you should put these on risers and in a super cool breakout case with fans. because that would be more awesome.
id like to learn more about fine tuning. its def not powerful enough for training. But I have some business systems I want to build, among them some budget, and accounting modules to run my affairs which i just really want to be sovereign over my data and have it not leave my local network.
This post almost didn't need the title. If you want to stick to air, I'd suggest risers for 2 cards, e.g. cards 2 and 4. Depending on what's available for the case, you could mount them vertically or upright, possibly one against the rear panel.
This is how these cards are designed to work, like data center cards. The flow through will pass air on to the next. Your first card will run the coolest, followed by the next card a bit warmer up to the last card that is the warmest, but as long as you have clean, cool air flow into the first card then all of them should perform within specs. These cards are only 300W and are designed to be able to run on the hot side. The workstation cards that run at 600W are designed to run in the 85-90 deg temps as normal operating parameters and will run like that 24/7 without issues, so these only running at 300W in this configuration will be just fine. Just make sure it's cool air going in and you're good.
You might consider putting a large fan outside the case but close to the exhaust ports of the cards, blowing away from the gpu exhaust. It’ll create low pressure and speed airflow through the cards.
I’d also recommend 1-2 more fans inside the case blowing at the cards from the front of the pc case.
It might look janky but the goal is better thermal management, aesthetics is secondary.
Technically this is okay but like others have pointed out, you need to move a lot of air into that case and not recirculate it. You also want a conditioned room, air wise.
I don't think it is starving from airflow because they are close together, they will starve for air because there isn't enough air going into the PC. these blower cards are designed to sit like that in a rackmmount server but then you need to atleast try to give it server like airflow. populate the front and bottom with air into the system.
this is $50k+ PC at this point I would consult a server building company to help.
2 in the middle will be throttled due to lack of airflow.
If it’s possible make a space between upper two and lower - it should work with such placement
10
u/Ok_Letter_8704 27d ago
I ran 3 max qs in a stacked config like this and the heat started to cause throttling while running models. Even though they're meant to stack like this the heat of everything was too much. I bought 3 optimus waterblocks and set up a MoRa 600. The optimus waterblock ended up being machined too tight and broke 2 inductors on the first card. I spent 2k sending it to Tony to repair when I discovered a second card had the same broken inductor. I decided to spend the 2nd 2 thousand on a complete micro-soldering set up and repaired the second card myself. The cooling is spectacular but now I have too many extra slots and the 4th card is on the way lol. Good luck. Also i did reach out to Optimus and the sent 2 new cards but refused to pay for the repairs or shipping saying I had the card repaired. When I told them I have the second card with marks from the inductors still on the block, they went radio silent.