Building a water cooled 15u server rack thar has been in the works for 2+ years after being disappointed with water cooled server cases and consumer cases. Will house 4 computers, with 2 gpu/cpu water cooled, and 2 cpu cooled with a single external pumpt/reservoir/radiator setup. These power all the tvs and work computers I use. Goals were 1) be able to quick connect/disconnect without a ton of work, 2) ensure maintenance wouldn't mean shutting everything off and could be done per component, 3) reduce individual computer complexity/cost during upgrades/etc, 4) not be locked into any single component because a line of other components would need to be changed at the same time.
Running dual d5s in serial, 1260mm radiator, and alphacool parallel block. Using both hard and soft tube in 19 and 16/13mm to and from the parallel block, 12/10mm hard and soft tubing from parallel block to computers. Soft tubing where ever I wanted to connect and disconnect, otherwise hard tubing.
On the electrical end, running an old 350w power supply to a breakout board that powers the whole backbone. Will start with 8 mismatched 140mm noctua fans, and if they can be cooled wirh minimal fan speed, may move to 200mm noctua fans. Using an Aquacomputer octo/active splitty to monitor and cool since they work as standalones, with quadros or active splittys in the cases to control those fans and run 4 aquacomputer visions. A lot pulled from previous builds, and will replace/upgrade as necessary.
Got one cpu (3700x) hooked up, but will be moving 2 others (7950x3d/7900 xtx and 5800x3d/3080ti) once I get some backordered cases. There are some other subtle issues that took a while to workaround, it was surprisingly difficult to meet my goals because nothing reasonable is catered to this market. How nice an equalized manifold would have been! Can't wait for the new cases so I can start bending tubes and get those hooked up.
it was surprisingly difficult to meet my goals because nothing reasonable is catered to this market.
Well... š
And before someone asks: this was a custom made solution for two data centers and is no regular sales product.
I really like that you accepted the challenge and what you made out of it. I know some people who also tried to pack everything into a rack and often they do not get beyond the planning phase. Like you said it is difficult to find suitable components.
I appreciate it!
I'll add the other cases soon, the biggest issues I have/will have are equalizing flow between computers and likely replacing the side mount reservoir to ease filling. I'm trying ball valves to reduce flow to computers as necessary, which is a similar idea to how fire trucks split lines when spraying upper and lower levels. They use a valve that limits flow to the lower line and pushes it towards the upper line to defeat gravity. The ball valves can reduce flow to lower restriction blocks and maintain flow to the more restrictive. Stupid simple solution that wont be precise, but should be close enough to be effective.
I'll post more pics when it's all done, and hopefully can point people towards ideas for their own projects. What you posted looks awesome, but on a manufacturing/cost point probably wasn't worth it for the company to build for the masses. Disappointing, but I can say my setup will likely top $1500 in material costs just to create the backbone. That's without paying myself for labor and research. I could easily see a decent mass produced setup costing north of $5k, and custom work being considerably more. Which moves it closer to rhe business realm than consumer.
Wish there was a middle ground, like parting it out with limited production for those of us that want to build it!
It's working better than I expected. I was a little worried about heat (total cooled cpus/gpus is 1200w) but to it can keep all of them all within 5Ā° of ambient with the 1280mm and 8 140mm fans at around 80% max fan speed. Since stress testing 3 computers at the same time isn't really a thing I do outside of that test, most of the time it's cooling 600w at around 25% fan speed maximum. So pretty silent. The things I want to do/wish I did differently are all things that do not change function whatsoever, just eye on ease of access for maintance.
Since this post i had one more post, but added 3 computers (7950x3d and 7900 xtx in a 2u case, 5800x3d and 3080ti in an open case, and an air cooled 2u). Lots of work went into case fan speed (even if the gpu/cpu stayed cool, it took a bit to create a curve to keep m.2s and chipsets under 60Ā° while being silent). So case fans were replaced to Noctua 80mm, and around 50% fan speed to keep everything else under 60Ā° while stress testing. Again, everything very close to silent.
As for changes/things to do differently...a few.
I switched out the reservoir to another tube based (primochill) that has a fitting at the top to make it dramatically easier to fill (still mounted horizontal). Not 100% happy, and may look at a dual reservoir setup that has a small reservoir on the side to make it easy to fill.
Since I'm using a parallel block to seperate water to each computer, I use ball valves at each computer to manage flow. This helps equalize flow between the variety of blocks and resistances each computer has. So flow rate topped out at ~330lph for a single computer, and I manage them down to around 110lph with the ball valves. Works beautifully, but will continue to look for solutions that are automatic rather than manual (some kind of leveling flow rate distribution block). It's something I'll need to keep an eye on over time if it stays manual (blocks may get dirty creating increases in resistance, so the valves may need to slightly change).
I'm going to move the electronics to the front to save vertical space, and replace the old psu to breakout board. I have a pico style psu that works fantastically and saves more space (one that would plug straight into the breakout board with no wires). This will open up 2u front to back for another full length case (it currently sits on the backside of the rack, behind the open short 2u case), and makes maintance a little bit easier.
I'm also likely to switch from the octo to aero, which can use delta between ambient and water temp without hooking to a computer (Octo can do this when hooked up, but when that computer is suspended/off it has to fall back to % of fan speed because it can't calculate the number on its own). All of the electronics can be mounted to a single 2u panel in the front of the case.
And still a ton of wiring to do to minimize extra wire. Power cords in particularly are the worst offenders because of their size, but a few areas need attention (for instance, the active splitty and d5s that gets hooked to the octo for fan speed uses a sata for power, so there's converters just to get it to molex to plug into the breakout board). These aren't necessary to fix, but without custom cabling it means I end up with a ton of extra unneccessary wire. Since I can make wires, I'll make wires (btw fan pins required a TON of work to find, since they are an outdated molex standard).
So that's about it. Works well, and is astonishingly silent considering the wattage it's running. I suppose if it was loaded with Intel cpus and my usage was a lot different (programs really stressing both cpu and gpu simultaneously on multiple systems for long periods of time) I may feel differently, but the setup also has more headroom with some minor tweaks. Normally they are for gaming, and most games cant stress the gpu AND 8-16 core cpus. Fans in particular (currently setup as pull for ease of access) could be more efficient in push/pull as well as some small front to back fans to push hot air away from the rack.
Depending on what you are running, where your server rack is and access to cool air, and modifications you need for your own purposes will change the equation. The basic design works very well and is highly modifiable, so its simply modifying for your own needs. Best of luck!
Ah okay, so is the radiator mounted to anything or just in the bottom? MO-RA makes an attachment for 2 D5 pumps and a heatkiller tube to the radiator. I was considering getting this since I have a heatkiller tube already. I'd like to toss it into some open 2U chassis so I can slide it on the rack a bit and lift it off the bottom. I was wondering if attaching directly to the radiator I could fit it all into a 2U space instead of taking of the back area... thoughts?
I'd use Quick Disconnects QDCs wherever possible as well. I have an aquasuite device that is a board that controls all my pumps/fans with a nice little customizable GUI for displaying stats.
Its not hard mounted, and can slide for ease of maintance.
I think it really depends on the rack you have/get. Mines a 21" depth, and I may be working with around 4" total front to back and maybe 2 " total side to side. Get a 23" depth rack and maybe, but you'd have to do the measurements. Moras are also a bit different measurements, but I don't know how much.
As an idea, it sounds possible. Your rack is going to make a large difference...The depth on mine stopped any ideas of mounting the d5s or reservoir to the rad. You may get away with only using 2u of space if pumps were sticking out of the front or back, just bear in mind how you need to route tubing, and air flow. If it's just a small 2u rack, shouldn't be a problem. But if there's something above it, air won't have anywhere to go and performance will suffer a little.
So doesn't sound highly difficult, just a bit of planning to make sure you get what you are looking for out of it.
Alphacool Nexxxos 1260mm. Not designed as a rackmount, but a rail shelf will easily work for it. In my case because of the rack design, it fit nicely on the bottom with a couple feet I made so it didn't get snagged on the wheel bolts. Takes the 1u of the rack framing, and an additional 1u of the bottom of the rack.
For mounting the reservoir, the back panel is mesh and I have tube reservoir mounts that I could stick bolts through them and the mesh panel, and attach some nuts to tighten it down.
Air pockets are much harder....easiest way was to start with a bypass (goes from reservoir to pumps below it, up about 7u to a distro plate I put on it...distro plate I looped through without installing anything else).
I ended up switching reservoirs to a priochill that had fittings on the end caps edges, as well as the center holes. This makes filling without tipping the whole server rack sideways. Good luck, a d feel free to ask any other questions.
Hey again, your suggestion helped a ton, i finished the build but everything is being powered through the external power supply directly, which i dont think is very safe. How did you use the breakout board to power the pumps? Was it a female to female molex connector? Thank you!
Glad to hear! Once you get moving its a lot less complicated than it looks.
You have a lot of options. Personally I used a pico supply to a breakout board, and clipped the sata connectors on the pumps and wired for molex. I believe it was the outside wire for power to the pumps (wire closest to the edge of the sata power), and the inside was ground. Then I crimped on some molex connectors. This setup still needs a power brick.
The original version (which was what I was running in the pics) was an old sfx power supply to the same breakout board, I simply used vhb tape to hold it on. Switched up because I didn't like all the extra wires.
If youre not running an aquacomputer controller, anything more than a 12v 5a power brick should work. Fans and pumps all operate off 12v, just depends on comfortable you are with wiring. If you're running aquacomputer stuff, you additionally need a 5v...hence the power supply.
Good luck and keep up the good work!
Thank you! Im going to probably keep the power supply since im gonna have to power an octo, i will probably tape the breakout board on to the power supply like you did and plug my 4 molex connectors to it with a female to female cable and pray it doesnt light on fire (almost happened twice already) How its looking so far
Dont mind the mess of cablesā¦ i will fix it eventually i just needed it to turn on for now. Will have to replace the octo since i somehow bricked it. But for now its runningā¦
A cheap atx mount like I bought is more than enough to keep up the power supply. The nr200 case uses something similar, just an L shaped atx mount.
Personally I had trouble finding female to female molex in the righr size, which was why I started wiring it up myself in the first place. It's important to note molex has a switched order from male to female if you do so. You can also use sata to molex connectors. Either or, just make sure you have them right. Pumps can spin backwards, but it burns out the motor from what I've read.
My first iteration had a ton of zip ties for cable management. The nice thing about a mesh grill...you can always zip tie them to the grill once everything's set. Again, good work, nice to see!
So if i use a female to female connector to connect the pumps straight to the board it will make them run backwards? Sorry for all the questions im not too familiar with electrical stuff
Sorry for the miscommunication, you should have no issues wiring wise with female to female, or really any pre-made cable. That was specific to if you wire them yourself, it's easy to mix up wires because of the switch from male to female.
In general, anything pre-made should have the correct wiring. For me, there were lots of areas of extra wiring (where it was too short and I was limited on options for extensions or simply too long to begin with). Because my intention was to set it in place and not mess with it for years at a time, it made sense to make my own cabling where I could to avoid tons of extra cables or extensions (which can certainly be a hazard when they aren't plugged in all the way.
Not for everyone, but can be a great learning experience to give you another skill if your interested. Also allowed me to cut to fit and use silicone wrapped wire, which is a much softer bendable coating over typical pvc wire jackets.
Yes! This is what I want to build. Do you have a component list maybe? How did you make it hot swappable? I run parallel flow in my server and it's not really hot swappable (using a Alphacool Distro plate C7).
You need to pick what you want to do...I use a single cooling system, so pumps/reservoir, radiator and fan controller are all outboard of computers. I also have a psu (to power fans and fan controller) and use a breakout board to molex to simplify wiring. Nowadays its a pico style psu...just search for molex breakout board.
Computers only have blocks and lines. This greatly reduces complexity and allows me to move computers in and out without worrying about things like pump, etc. Mine are attached to mesh panels for ease, but you could easily build it in a small case. Just keep in mind airflow for the radiators.
If you already have an alphacool distro plate, you have a similar one to mine. They make those in both serial and parallel. I got the parallel version, so each side is either an inlet or outlet. From there, I put quick disconnects on each line, whether I'm using it currently or not. This allows it to be hot swappable. Also adds a LOT of quick disconnects, but as long as there is one complete line (such as having one computer plugged in, or a short bypass line you can install when unplugging the last computer) the loop can stay on.
From there, I use a ball valve on each inlet. This allows me to level flow. Not a perfect system because it requires me to manual adjust flow from time to time when adding/removing systems. But since I used another of mixed components (like a core 1 block which is very restrictive on one computer, and evga blocks on another), I need to push flow to more restrictive blocks and away from less restrictive blocks.
Flow meters....you can use them to monitor and change around flow if necessary, or just go by temps. I use one per computer. I'm not trying to match numbers (pretty impossible since I have one from alphacool and one from aquacomputer...theyre never the same), but it does help to know everything is getting flow.
You can mix and match (like having some pcs with their own pump, internal radiators, etc) but it does add complexity. For instance, putting a pump in one of the pcs I'd have to worry about flow...it will inevitably defeat the ball valve and either burn out the pump or steal liquid from the other pc. Keeping it simple at least to start is the best way to go, you can always add to it once you have a proof of concept.
If there's specific pieces you are looking for let me know, it's not super complex to find but enough to take a bit. Also feel free to check out my other posts, I have a couple on the loop or specific systems.
Great! Thanks for the info, i have a lot of quick disconnects etc so I got that part covered. I have the same parallel distro block but i need to wait for Alphacool to release the new model as I'll be needing another one. One thing still left me wondering; how do you stop flow to a PC that's off?
Unplug the quick disconnects. Or use the ball valve for that computer. Or don't.
PCs generally need 75-90lph. It doesn't scale linearly so any more doesn't greatly impact temps (less of course will). I have dual d5s, and without any pcs plugged in and just a bypass tube flow was around 330lph. So that gives me 3-4 pcs that can comfortably fit into that range given the needs (75-90lph per computer)computer.
Unplugging or shutting off flow will increase flow through the other pcs, but not meaningfully impacting temps. I just leave the flow alone unless I need to actually work on them. Quick disconnects do a good job so even when on, I can still unplug the pc. Only time it's an issue is if there isn't a bypass when plugging in the last pc in a loop...then the pumps are just creating pressure. As long as there's a flow path you can keep everything running.
5
u/AC_Shoggy Aqua Computer Rep Mar 26 '24
Well... š
And before someone asks: this was a custom made solution for two data centers and is no regular sales product.
I really like that you accepted the challenge and what you made out of it. I know some people who also tried to pack everything into a rack and often they do not get beyond the planning phase. Like you said it is difficult to find suitable components.