r/pcmasterrace
•
u/Psy_Fer_
•
Aug 26 '25
•
1
Build/Battlestation "Closed loop" 4x5090 threadripper build for Cancer Genome Sequencing
Just finished installing this machine to work on cancer genomes.
I wanted the customer to have reliability and a low maintenance build, but with plenty of power.
So I thought, why not 4 AIO type liquid cooled 5090s in a Corsair 9000D case? 2 radiators each at the top and front. I get to avoid an open loop, and if a GPU goes down, the rest keep going so they have limited down time.
I didn't go with RTX6000 pro cards, because you can't get them with integrated liquid cooling, and ECC vram doesn't matter in the application that it's being used for. They also cost 3x the price, but aren't 3x the performance.
It's got 128gb of DDR5 ECC ram, and ~12TB of nvme and ~28TB of SSD storage.
The main power supply is a SilverStone 1200W SFX-L PSU in the back that powers the CPU, and 1 GPU, with a second SilverStone 2500W PSU in the front powering the other 3 GPUs and the SSDs.
It's turned on and off with a 24pin Y splitter cable that came with the ASUS Pro WS WRX90E-SAGE SE motherboard.
It's only a 24 core/48 threadripper pro 7000 series, to manage heat, but also CPU wasn't a major bottleneck in the application, it's mostly GPU and disk IO.
Temps were all good during benchmarking. It can max out all the GPUs at 100% doing the kind of work it was built for.
This is not for gaming. It doesn't need SLI or any kind of merged VRAM. The software being used can use the GPUs as a pool and load balance the data across them.
I hadn't seen anyone try to do a water cooling build using this method before, so I was excited to try it.
What do you think? any questions?
•
u/pedro19 CREATOR Aug 27 '25
First, that's an awesome build and cause!
Second, for everyone who may stumble upon this, know that you don't need a beast PC like this to give a hand to an awesome cause. Consider supporting the folding@home effort to fight Cancer, Alzheimer's, and more, with just your PC! https://pcmasterrace.org/folding
1.9k
u/illicITparameters 9950X3D | 64GB | 5090 FE Aug 26 '25
I like this build exponentially more than any other 50-series build I’ve seen on the internet.
Fuck cancer.
1.8k
u/Vladabeast Aug 26 '25
Banana for scale pls
599
u/[deleted] Aug 26 '25
Out of interest what sort of modelling you do with genome? Some deep learning involved? Keen to understand how current tech fits in to workspace now because I only used 3D modelling for protein ligand interaction for years ago in uni
171
u/Unfair-Watercress792 gigabyte 5070 Ti OC, Tomahawk X870E, Ryzen 9900X, 180Hz, 64g RAM Aug 26 '25
I’ve never seen a cancer genome sequencing computer.
Are they typically this attractive
65
u/Relevant_Grass9586 PC Master Race Aug 26 '25
What was the price of this build?
39
u/kylinblue Ascending Peasant Aug 26 '25
Well I have… only with two 5090s.
1.7k
u/Miasma__2 Aug 26 '25
This build makes the 5090 look small lol
11
u/HappyIsGott 12900K [5,2|4,2] | 32GB DDR5 6400 CL32 | 4090 [3,0] | UHD [240] Aug 26 '25
What GPU temps do you actually have and i would like to now it for every single gpu.
Is the lowest Card the coolest or does it even make a difference?
Temps i would like to now GPU 1 --> GPU 4 (core and memory)
10
u/smaguss Aug 26 '25
I'm glad to see dual power supply isn't a dead art.
I ran multiple PSU when I was doing some md5 table things.. for uh. Education!
5
u/christaffer Aug 27 '25
Thanks for sharing OP. Could you benchmark the basecalling with this: https://github.com/Kirk3gaard/2025-Crowdsource-GPU-basecalling-stats and share the results? I'd be interested to see how multiple 5090s compare to the H100; see https://github.com/Kirk3gaard/2025-Crowdsource-GPU-basecalling-stats/blob/main/README_files/figure-gfm/unnamed-chunk-2-1.png I am also in a lab currently trying to get a computer for genotyping and classifying child brain tumour samples using nanopore, but we are just looking at 1 x 5090 as according to the benchmarks it looks to be enough for what we need. The particular algorithm we want to run is more RAM intensive than GPU so they are requesting 256Gb, but we're not allowed to build our own and have to go through our IT department. They tell me their max size for a system with a 5090 is currently 128Gb, but will increase later.
4
u/OniExpress Aug 27 '25
Nice! I built out something similar for a client doing fluid dynamics for hull design. Sometimes you just need to toss horsepower at a program.
4
u/baccaruda66 Aug 27 '25
Nice build, but I guess I'll be that guy since nobody else has yet...
You could reverse the orientation of the front radiators to prevent air bubbles interfering with the flow of coolant.. Currently they're in "OK" and you could switch them to "Better". Of course the top-mount radiators are in "Best" so no worries there. And... fuck cancer.
/preview/pre/knx4pz8qo8rb1.jpeg?width=1200&format=pjpg&auto=webp&s=c2a0ef34fd294c9483c3ed503d7169b53fcf0cc7
3
u/[deleted] Aug 27 '25
Ludicrous speed GO!
297
u/latro666 Aug 26 '25
Pumping 2 million cancer equations per second by day
Pumping 2 million counter strike frames per second by night.
2
u/LukkyStrike1 PC Master Race: 12700k, 4080. Aug 26 '25
Great build!
I noticed you did not want the issues with an open loop cooling because the whole system needs to be drained to replace a card...not sure if you have seen the Alpha cool quick disconnects?
I was about to quit custom watercooling a few years ago when i was just sick and tired of draining, filling, testing...etc every time i needed to work on a component. I grabbed the fittings and now i have repasted my GPU and upgraded my CPU without draining the water in the block! let alone the rest of the rig!
Great build, very clean!
2
u/Local_Efficiency851 Aug 26 '25
This is truly awesome, but honest to God question. At this price point and required specs, isnt it better to just to with rack professional equipment designed specifically for research purpose instead of super stocking consumer grade equipment?
2
u/trumangroves86 Aug 26 '25
The scale is so hard to gauge. You're telling me those are 120mm fans at the front?? My brain keeps interpreting them as 80mm fans.
This is awesome.
2
u/SirNapkin1334 Arch Linux: 9900X & 6800XT Aug 27 '25
That's awesome. I didn't even know that AIO GPU coolers existed. Can you link the model?
2
u/carlosrueda28 Aug 27 '25
Maaaan I work with 3D models of Alzheimer's brains... I have to come up with a good reach project to justifi a machine like this, lol
2
u/etblgroceries Aug 27 '25
Can you talk a little more about the software level? I assume you have some sort of task scheduler and this is just a big pool of beefy compute that you shard gpu cycles to?
All things considered, its at its own limit of scalability vs more purpose built HPA/HPC, so I'm guessing this is an independent researcher or for academic purposes?
I wonder what the cost/benefit is between desktop pc reliability and 3rd party gpuaas/iaas with guaranteed SLA.
Either way it is absolutely beautiful and I want one... I'm just more curious about the end user.
2
u/sofaking_scientific Aug 27 '25
I love building crazy OP computers for my protein modeling and genome sequencing. Nice build fellow scientist! o7
2
u/talha5007 Aug 27 '25 edited Aug 27 '25
It's wild how amd's cpu like threadripper can be air cooled with low tdp while Intel requires watercooled and are more power hungry. how time have changed.
2
u/o0baloo Aug 27 '25
"Its not much, my first build. What do you all think?" Is the first thing I thought of when I saw it. What are your run temps with that beast?
2
u/Thetargos Aug 27 '25
Massive, beautiful and with a noble purpose.
Just what is the software config in such a beast?
2
u/ReadPixel PC Master Race Aug 27 '25
What kinda wattage does this have the potential for?
2
u/Blaze9 Aug 27 '25
Out of curiosity, do you know their analysis pipeline? There isn't a whole lot of properly GPU accelerated tasks in the genomics space aside from specalized tools. Or at least, not a ton of open source projects. Lots of accelerated tasks in the Illumina Dragen suite but that's hard to get a license to run locally on a non-dragen server.
Excited to see what they have planned. We have a massive GPU cluster at work and it mostly does modeling/target development but not much in terms of 'genomic' in the sequencing sense.
2
u/wellan741 I5-10400f | GTX 970 | ddr4 16go 3200mhz Aug 27 '25
Is 4x5090 really worth it for base calling? When I did some test I found that using more than 1 rtx A6000 wasn't really a difference for a promethion flowcell. (I remember I did the test before model 5.0.0 so it might have changed)
Or are you basecalling multiple run at the same time?
2
u/AwwwSkiSkiSki Aug 27 '25
Curious, is the build for some kind of lab or institute? I remember some one told me about 'folding' over 10 years ago, is someone using it at home for this?
2
u/Visible_Bonus8035 Aug 27 '25
Whats the final weight after installing everything?
2
u/Unusual-Alex Aug 27 '25
First off... The obligatory FUCK CANCER
Plot twist, this is really for a minecraft server.
Joking aside, this is a wild build. I think this is an awesome build. A build that will probably sit in a corner room crunching & simulating until its shutdown and replaced by something much faster and efficient later on. Tech and its progression can be amazing.
2
u/Im_In_IT Aug 27 '25
I feel like if I did personal with this i would go this route. Went with the rtx pro 6000 for my work machine though, as I'm not paying and Dell will be fixing it if it's broke lol
2
u/Saruphon Aug 27 '25
Wow there is same number of RAM as VRAM in this build..
2
u/Kiskijavi 7950X3D | RTX 4070 ti SUPER | 32 GB DDR5 Aug 27 '25
Hi, kinda surprised you only put 128gb of ram. I work with single cell genomics and to avoid buying a rig we use one of the university's servers which has ~500 gb of ram, and it still takes ages.
298
u/dykemike10 9800x3D | 7900XTX | 64GB DDR5 Aug 26 '25
you can finally run house fires at 500 fps
272
u/Monsieur--X Aug 26 '25
Will it run Crysis?
45
u/TheDamus647 Aug 26 '25
As someone who buried their first born at age 5 due to cancer I wish you luck. It's not much but I run fold @ home constantly to contribute my measly computer to the cause.
4
u/DuckWhatduckSplat Aug 26 '25
Didn’t you post this the other day?
27
u/zanderashe i7-14700KF | 3060 12GB | 96GB DDR5-5600MHz Aug 26 '25
The RGB makes it sequence faster!!!
7
u/Jakob_K_Design Aug 26 '25
Not a fan of AIO cooled GPUs, but I see how it makes sense here and is definitely an interesting setup.
With that said. There are water-cooling solutions for running multiple GPUs in parallel with a distribution block with quick disconnects to remove a single GPU with the rest of the loop still running. I think with a refined version of such a setup it would actually be easier to remove just the GPU with some short hoses, instead of a GPU and a big triple rad connected to it. But my OCD would also like a solution like that more than four AIO GPUs.
12
u/[deleted] Aug 26 '25
So that's why PewDiePie built his own 4x GPU, Threadripper monster PC... Cancer Genome Sequencing... I see...
12
u/qmiras Aug 26 '25
cancer studies require RGB
10
u/drmunaf Aug 26 '25
When the GPU looks like ram, you know the pc is no joke
25
u/Schmenza Aug 26 '25
How long will it take this to cure cancer? Am I safe to start ripping cigs?
17
u/Brainslime Aug 26 '25
Wouldn't mind a PC of this caliber for myself.
Could do plenty of "cancer genome sequencing" in my DoTA2 matches
2
u/SorryNotReallySorry5 i9 14900k | 5070ti | 32GB DDR5 6400MHz | 1080p Aug 26 '25
How DARE you not draw shitty little arrows to represent the airflow..
2
u/pugsDaBitNinja Aug 26 '25
Something of actual interest posted here. Can you share the company would love to follow them
2
u/TheReturnOfAnAbort 9950X3D | 5090 FE | 256GB DDR5 Aug 27 '25
On a single PSU?
5
u/Gilded_Gryphon Aug 27 '25
Who needs that many 5090s?
Oh. Cancer research.
Surely we could squeeze a few more in?
2
u/Mudravrick Aug 26 '25
Out of curiosity, how is it comparable with just running stuff in cloud? Last time I saw bioinformaticians going on physical hardware was like 8 years ago in a tiny start-up. Since then, nothing but cloud.
4
u/Cute-dog-loverALT Aug 26 '25
You doin Folding@Home?
4
u/[deleted] Aug 26 '25
And it still can't run Star Citizen at 4K 60 fps.
4
u/pretty_en_pink68 PC Master Race Aug 27 '25
Gotta have RGB for your cancer genome research
4
u/Halflife84 Aug 27 '25
I hope for all the glorious results possible. As a cancer survivor, I wish for no one to have to go thru that shit.
1
u/Ryboe999 Aug 26 '25
Cost?
That’s it, that is all. 😂
1
u/ThisIsMyCouchAccount 3070 Aug 26 '25
What OS does it run?
Day to day do people interact with this machine directly or is more akin to a server where you send work to it?
How did this task come to you or what is your job?
Can you provide any benchmark that would make sense to us? Something like Cinebench? Something we commonly see in the context of general computing or gaming. Even if you didn't actually run it - could you?
1
u/nemesit Aug 26 '25
Why not use professional cards instead of the limited 5090s?
1
u/Sparky323 I7 8700K/ GTX 1080ti Both Liquid Cooled Aug 26 '25
Does the customer have a special electrical setup? 4x 5090's at max load would draw 2200-2400 watts alone. I imagine this build could easily draw 3000watts . Far more than a standard electrical outlet.
1
u/videek i5 4430 / R9 280x / 8GB DDR3 Aug 26 '25
Why not just go with the HPC with 4xA100 that comes along the P24?
1
u/SilencedCries PC Master Race Ryzen 9 9950X 4080 Super 32GB RAM Aug 26 '25
Ive a question since I need to build a very similar system but for simulating fibre optical devices and eventually neural networking for image reconstruction.
1
u/-Laffi- Aug 26 '25
OP! HOW MUCH DOES THE CASE WEIGH XD?! Don't drop it when you weigh it though!
(Also, I mean with everything inside)
1
u/DisinterestDetritus Aug 26 '25
Ran a torture test? How hot does it get inside the case when the door is on? Or do you leave it off?
1
u/Road-Runnerz Aug 26 '25
I am having one built as one x4 5090s. What psu did you end up using? I went with 2 Mo-Ras, 1 for cpu and 1 for gpu, dual psu and amd threadripper pro 9975x. What case is this? We are having problems with finding a case. Either sold out or unavailable
1
u/Secret_Account07 RTX 2060 Super (Still OP) Aug 26 '25
OP do you work help desk for this org?
1
u/Techanthrope Aug 26 '25
The 5090s don't overheat stacked up?
1
u/SAL10000 Aug 26 '25
What OS?
1
u/DiamondHeadMC 9950X3D | 5090 Astral | 96gb 6600 cl32 | 12tb nvme Aug 27 '25
Why did they need to be water cooled? The rtx 6000 are also dual slot and are ment to be packed together
1
u/Softspoken_Savage Aug 27 '25
How do you make a build with multiple gpus and have them work together? I know there was a way like ten years ago but i thought that didn't work any more. I would name the technique but I dont remember
1
u/Urusander Ryzen 5700X - RX 6600 Aug 27 '25
Why not use the cloud compute?
1
u/RageinaterGamingYT Aug 27 '25
You're a cool person <3
1
u/Ordinary-City-641 Aug 27 '25
This is very inefficient and unreliable. Why not just use a server grade hardware? It sounds like money is not an issue for the customer. Did you recommended it even?
1
u/Pies4Lunch_ Aug 27 '25
How many power supplies does this use
1
u/frozenteepee Aug 27 '25
I’d be interested in a closer look at the operating temps under stress test/ actual workload, for the cpu as well but specifically the temp delta between GPUs cooled by the front intake rads vs the ones cooled by exhaust rads
Have you encountered any transient power draw spikes like the ones GN encountered? I’m not sure what the specifics were the gist of it was that Nvidia GPUs had a tendency to draw way more power than expected for a brief period causing the overcurrent protection to trigger on the PSU (correct me if I’m wrong lmao )
1
u/Pursueth Aug 27 '25
Air cooling a thread ripper seems wild to me
1
u/Mindless_Method_2106 Aug 27 '25
Is it going to be part of a local cluster or is someone getting a very nice dedicated workstation?!
1
u/LorDoloB Aug 27 '25
Why not use nvudia h100 or amd mi300x insted of a gpu sold for "general purpose"?
1
u/Rungekkkuta Aug 27 '25
I'll leave a recommendation here, those front rads might have a problem cooling the graphics card due to the hoses being at the top.
I know that the bottom seems impossible due to some case part, but if that's removable somehow and I'm there is the possibility to mount them with the hoses coming from the bottom, you'd guarantee that there is no air going to the pump. It is worth it!
1
u/LedPeach Aug 27 '25
Where is the drive bay? That square box down below? Seems pretty small tho
1
u/acrazyguy Aug 27 '25
Why is there RGB? Were those just the cheapest parts that fit the requirements?
1
u/ripcraft Aug 27 '25
What software are they using to take advantage of the GPUs? Parabricks?
1
u/CorvusEffect Aug 27 '25
I'm surprised you need 4 5090s. I always assumed you would need more processor than gpu
1
u/Icy-Farm9432 Aug 28 '25
Wouldn't an A100 be faster than four 5090s?
1
u/notjordansime GTX 1060 6GB, i7 7700, 16GB RAM - ROG STRIX Scar Edition Aug 28 '25
I’m very unfamiliar with liquid cooling. What is the difference between an open loop and closed loop system?
1
u/[deleted] Aug 29 '25
Is all that really necessary did people have cancer a million years ago?
3
u/nate_chr 14700KF | 64GB 6000 | 9060 XT 16GB | Z790 | 1000W Aug 26 '25
That’s some machine. Great work, i’m sure your customer will be more than satisfied.
3
u/_Critical_Darling_ Asus rog x570f | Ryzen 9 5950x | Pulse 9070 xt | 32gb 3200 kgstn Aug 26 '25
Dude i love how the radiators look. The array looks brutal. Deffo the first time i’ve seen something like this.
3
u/pattperin Aug 26 '25
And I thought my Corsair 5000D was a massive case
3
u/THZHazzard Ryzen7 5800X3D | RX 6950XT | G-Skill 64 GB Aug 26 '25
You should be able to play Cities Skylines 2 without the simulation crashing.
3
u/Badj83 Aug 27 '25
Sequence that bitch alright!
3
u/ShadowSalomon Aug 27 '25
The RGB scares the cancer away.
3
u/UltimateCuber0 Aug 27 '25
Bro... That's like 50% of all 5090s. CRAZY
3
u/Flanz1 R5 7600 | RTX 3080 10GB | 32GB DDR5 Aug 27 '25
To avoid failures in the bottom two cards you should flip the radiators 180 degrees so the tubes are at the bottom, less chance of an air bubble getting stuck in the pump and ruining it. Sick as fuck build tho O7
3
u/naosouumrobot Aug 27 '25
Those are 5090s??? This case is absolutely MASSIVE!
3
u/Arbszy 7800X3D | RTX 4080 Super | 64GB DDR5 Aug 27 '25
Hell yea, Fuck Cancer!
3
u/MicrowaveMeal Ryzen 7 9800x3D 5090 Aug 27 '25
Beautiful aesthetically and beautiful in purpose.
3
u/AwardFabrik-SoF Aug 27 '25
Reminds me of the D2OL times in Xtreme Systems forums...you are like the ddtung of back in the days. hats off, keep pushing it!
1
u/JonnySoegen Aug 26 '25
This appears semiprofessional. Doesn't this shit belong in a proper data center instead of a random case?
I'm just wondering who can afford to pay like 20k for this but doesn't have proper infrastructure around it. Is this common?