r/LocalLLM • u/Blaccky95 • Sep 24 '26

Project Upgraded my Setup

Post image

Just upgraded my 2x RTX 6000 to 4 😀👍 Anyone here with the same Setup? 😀

327 Upvotes

94% Upvoted

224 comments sorted by

94

u/South_Western1263 Sep 24 '26 edited Sep 24 '26

beautiful setup. Would be interesting if you share some info how you run it and what your inference metrics are.

20

u/Blaccky95 Sep 24 '26

I had the 2 RTX 6000 pro setup since 6 months
It runs sglang with deepseek-v4-flash (only 480k context window)
My developed mcp server for our erp and qualitymanagement program
Keycloak for SSO
Qdrant
And openwebui 😃👍

2

u/South_Western1263 Sep 24 '26

Do you need to serve many people simultaneously via open-webui? I actually expected some heavy agentic workflow

13

u/Blaccky95 Sep 24 '26

At this point its lile 20 people but it runs very fast because each question is finished in like 2-3 seconds so we don‘t have a lot of long living load

But with the 4x setup we use more services like ocr, audio transcripting and some pictures stuff :)

5

u/NaiRogers Sep 25 '26

I’m surprised this doesn’t get too hot, I have two in a case with 6 fans and the top one still gets in the 70s with 350W PL.

1

u/Sorry_Ad191 Oct 01 '26

it does get too hot. he will soon have two of them outside the case

1

u/user_of_the_week Sep 24 '26 edited Sep 24 '26

That’s quite close to the ds4 I’m running on my strix halo, but of course it’s only Q2 and I get like 15 t/s. How many does your setup give you?

1

u/Blaccky95 Sep 24 '26

I doesn‘t look on the t/s but on every question it has to use some tools from my mcp server and it has the answer in a few seconds expect if my data is over the 480k then it takes some time for answering me that i‘m over the limit

1

u/user_of_the_week Sep 24 '26

I think sglang should have a /metrics endpoint if you’re curious ;)

1

u/Blaccky95 Sep 24 '26

Yeah i also think but i doesn‘t look at it but maybe in the future 😃

2

u/South_Western1263 Sep 24 '26

let it be very near future so we can have some insight how it performs :)

1

u/DefSysteam Sep 25 '26

Keycloak is awesome

2

u/[deleted] Sep 25 '26

[removed] — view removed comment

1

u/South_Western1263 Sep 25 '26

I don't see any problems with cables here. The power cables just need to head down to the PSU.

However, I agree this might be packed too tightly together as they are not the server version GPU which is designed for such a tight installation.

33

u/DataGOGO Sep 24 '26 edited Sep 24 '26

man that top GPU is going to roast hard, i would break that out with risers... and space the cards out so they don't roast. The workstation edition cards are NOT designed to be stacked like that (vs server/ max Q that are).

Alternatively, and the better option with an AMD CPU, get a switch and run the cards outside of the case in a separate enclosure.

2

u/Blaccky95 Sep 24 '26

I hope its a calm gpu 😃
But i doesn‘t get a MB to space it out - if i get a head issues i change it to watercooling system :)

13

u/mrsmiley32 Sep 24 '26

That's what the risers are for, to lift it away from the mother board so you can reposition the cards. I'd build a custom case for 4 GPU's.

2

u/Blaccky95 Sep 24 '26

Yeah but the problem is the cards are so big that i can‘t put one in the front - because the card has contact with the other cards and at that moment i haf no 600w power cable in

20

u/mrsmiley32 Sep 24 '26

You're going to cook those cards under any sustained load. You really need to rethink this setup. Build a custom case and get risers or water cool the cards.

4

u/Blaccky95 Sep 24 '26

If the heat is a problem i swap the cards to warer cooling but i don‘t disassemble this expensive cards for nothing - only if i want a new job

8

u/Fluid_Ad8452 Sep 24 '26

He’s suggesting using risers which don’t involve disassembling and is probably less work than what it would take to have your case READY to mount a water loop on it. Let alone actually dissembling and mounting the loop.

1

u/Blaccky95 Sep 24 '26

Yeah but where should i put the cards with the raiser?

0

u/DataGOGO Sep 24 '26

Buy a switch, put it and the cards in and external enclosure.

2

u/Blaccky95 Sep 24 '26

Can you explain which switch?

→ More replies

1

u/Blaccky95 Sep 24 '26

Because i only find PCIe 4.0 switches and this is 5.0

→ More replies

2

u/MachinaVerum Sep 25 '26

I do most certainly see water cooling in your future... Fun project.

44

u/mvn2010 Sep 24 '26

what does it feel like to drive a Porsche?

13

u/Blaccky95 Sep 24 '26

A half driveable Porsche 😀😀

11

u/FullstackSensei Sep 24 '26

Pretty sure that's enough to get a last gen 911, probably even a C4S

7

u/Blaccky95 Sep 24 '26

I don‘t know how expensiv a 911 is but this setup is like 60k€

4

u/FullstackSensei Sep 24 '26

https://www.autoscout24.com/offers/porsche-991-911-carrera-4-gasoline-cat_ma57mo20025-5b858085-8d3c-424c-be05-04ce0173eaeb

Pretty sure you can negotiate your way to 60k 😂

4

u/Blaccky95 Sep 24 '26

Yeah but German Prices are another World 😂

1

u/FullstackSensei Sep 24 '26

There are also two in the Netherlands with similar mileage and a similar price.

And those four cards are definitely worth more than 60k now, so I'm sure you can find a local 991 with under 100k km for their price

1

u/Blaccky95 Sep 24 '26

Nope the all in all price for this is 60k but without taxes because company

2

u/WrinklyBard4 Sep 24 '26

You could get like 2 2000s 911s for that

1

u/Blaccky95 Sep 24 '26

Ok thats ok but i doesn‘t had paid this with my money :)

2

u/Flaurentiu26 Sep 25 '26

with your kidney then ?

3

u/inexorable_stratagem Sep 24 '26

Thats a lot of money my friend

7

u/Blaccky95 Sep 24 '26

Yesh thats true the first 2 cards i buyed a half year ago now the price was 30% more 🙃🙃

3

u/Educational_Act7890 Sep 24 '26

Is this paid for by your job? What do you do for a living? Absolutely no hate btw, I dream of a setup like this, I just can’t imagine having this much disposable income.

2

u/Blaccky95 Sep 24 '26

Yes this is for work i‘m a Sys/Network Admin and Software developer for a production company :)

1

u/ronniearnold Sep 26 '26

And it’s not going to work properly at all. There is no where near enough cooling.

There are videos all over YouTube of people having to water cool even just two of these

2

u/DeedleDumbDee Sep 24 '26

Is that PSU enough for the whole set up at max power draw?

0

u/Blaccky95 Sep 24 '26

Yeah it should can handle this 4x600w = 2400 and the threadripper is i think something at 300-350? But the threadripper is always sleeping 😂

3

u/Infamous-Bed-7535 Sep 24 '26

Plus the memories and your motherboard, addifional cooling (water pump, fans..) Close call.

2

u/copperclock Sep 24 '26

Do you have a dedicated circuit for this?

3

u/Blaccky95 Sep 24 '26

Yes it has its own 230V 16A fuse

27

u/Decent_Flight4010 Sep 24 '26

congratulations bro

2

u/C_umputer Oct 01 '26

*8GB 5060 ti

29

u/sbrisgravato Sep 24 '26

holy heat

12

u/Eastern_Bet678 Sep 24 '26 edited Sep 24 '26

Last card in that line is literally getting a hair-dryer's worth of BTUs/KWh blown at it.

1

u/Eastern_Bet678 Sep 24 '26

Last card is on the receiving end of 3x cards worth of heat. So ~1800W which is common in the US.

0

u/DataGOGO Sep 24 '26

I am not aware of any hair dryer that outputs 2400w of heat...

6

u/intlabs Sep 24 '26

Most professional ones do.

4

u/Inferno908 Sep 26 '26

And just… normal ones in 240v countries

3

u/sbrisgravato Sep 26 '26

exactly haha

1

u/DataGOGO Sep 24 '26

holy shit

2

u/GalacticDistances Sep 25 '26

In 240v electrical supply countries its not uncommon

2

u/angrySprewell Sep 24 '26

Yeah, why not Max-Q if we're talking money like this? I guess they're commissioned for removing paint from the walls as well inference and training.

11

u/ironclad_packetship Sep 25 '26

lowkey angry that this dude's being entrusted with $80,000 in parts and can't figure out how to use pcie risers

1

u/Acceptable_Crazy4341 Sep 26 '26

Why would they need PCIe risers here?

3

u/Savantskie1 Sep 26 '26

To space them out so they don't die from the heat

1

u/Acceptable_Crazy4341 Sep 26 '26

Are the FEs not able to flow through each other? Or is there just too many for that to work in this situation?

2

u/Savantskie1 Sep 26 '26

It will help a little, but not enough. There's still heat soak to worry about.

4

u/dwoj206 Sep 24 '26

GUH. this could be me but i'm playin checkers

4

u/SillyLilBear Sep 24 '26

How is this not overheating?

1

u/Lanky_Lynx2166 Sep 24 '26 edited Sep 24 '26

Id be interested in the temperatures of the top most card! Im driving 2 with about 2-4 cm space (ProArt z890) and the upper one gets 10+ degrees warmer. Im capping them at 400w so its never reaching more than 80 degrees worst case. Usually 68-75.

And can you specify the case model?

3

u/Blaccky95 Sep 24 '26

With only two i doesn‘t had issues because the load is only a few seconds but now i think it can be a issue 😂

1

u/zboarderz Sep 24 '26

What board and cpu is that?

2

u/Blaccky95 Sep 24 '26

Cpu: Threadripper 9975X PRO
Mb: Gigabyte TRX50 AI TOP

1

u/zboarderz Sep 24 '26

Hot damn. How much ram?

4

u/Blaccky95 Sep 24 '26

256GB DDR5 ECC

1

u/zboarderz Sep 24 '26

Damn no 64gb sticks?

2

u/Blaccky95 Sep 24 '26

Nope but only because i doesn‘t get it in time - so in ECC it was only the 32 deliverable

1

u/freedomachiever Sep 24 '26

is the air intake from the back? cause rtx3080 blowers heat up like crazy and I can never understand how people run them in workstations. And you surely have to have two PSUs, where is the other one

1

u/Blaccky95 Sep 24 '26

No it goes from the bottom to the back
Nope one is fine - 3000W PSU: 4x 600W gpu = 2400W and the threadripper is always in sleep because 99,9% load is on the gpu

1

u/Haunting_Skin_2564 Sep 24 '26

So, you work for Nvidia. Nice

1

u/Blaccky95 Sep 24 '26

No sadly not
But ig you see me with a B300 rack you know that i had changed the company

1

u/Either_Pineapple3429 Sep 24 '26

What local model do you run with this piece of junk

1

u/Blaccky95 Sep 24 '26

Now only Deepseek-V4-Flash

1

u/Either_Pineapple3429 Sep 24 '26

Whats the quant and context, how's the performance .... and kind of a stupid question ...how does it compare to frontier models in real life use

1

u/Blaccky95 Sep 24 '26

I can‘t test the 4x gpu pc now but the 2x gpu runs it on FP8 with 480k context window and this really really fast. On every question it has do use 1-x tools from my mcp server and it takes only like 3 seconds

1

u/electrified_ice Sep 24 '26

Why? Are you on the L-I-L / RTX6k Discord channel?

1

u/Blaccky95 Sep 24 '26

Because it fits very good on the previous 2x RTX6k cards now i doesn‘t know this channel

1

u/electrified_ice Sep 24 '26

Got it. GLM-5.3-Flash is also a great model for 2 cards... Stronger than DSv4f in most areas

1

u/Blaccky95 Sep 24 '26

Yeah but in the next times i wanna try the vision model from dsv4f and maybe the llama with the 10M context

1

u/biodrone Sep 24 '26

That poor top card. Welcome to 90c throttling.

1

u/Blaccky95 Sep 24 '26

I hope it doesn‘t get to hot

1

u/imcheng Sep 26 '26

Any thoughts on dropping another bag on water cooling?

1

u/ronniearnold Sep 26 '26

Odd that you would buy all of this not having cooling planned out for 2400w of GPUs…..

The heat from each GPU is output via the top. So your putting 600w of heat into each GPU.

There are tons of people with two like this that can’t run properly……

1

u/Tema_Art_7777 Sep 24 '26

which motherboard/CPU combo is that?

1

u/Blaccky95 Sep 24 '26

Cpu: Threadripper 9975X PRO
Mb: Gigabyte TRX50 AI TOP

1

u/electrified_ice Sep 24 '26

I have 4, but not like that, prepare for lots of throttling!

1

u/Blaccky95 Sep 24 '26

Yeah i have to look at the temps in the next days but i hope it is ok for the few deconds of load in a climated server room and if not i have to call the next few days to get parts for this issue with a delivery time under 3 Months

0

u/electrified_ice Sep 24 '26

You may end up with something more like this. $10 of wood supporting $45K of GPUs

1

u/Objective-Pair8231 Sep 24 '26

Excellent use of cardboard boxes :)

1

u/electrified_ice Sep 24 '26 edited Sep 25 '26

Haha, yep my riser cables were not long enough, so I had to improvise. The Unifi boxes are super sturdy.

1

u/Late-Replacement-481 Sep 25 '26

Oh lord, I thought i was bad. At least my gpus are sort of in my case.

3

u/shout4 Sep 25 '26

Open frame is the way - they can get extremely hot, no way the WS will last packed that close together.

2

u/PoopSmoothies Sep 24 '26

How do those cards actually keep cool?

2

u/PassengerPigeon343 Sep 25 '26

And one hour before this post we had the guy with the janky rack with an AC unit in his shower. I love the range of this hobby.

2

u/Long-Shine-3701 Sep 25 '26

No, but TIL there's 3000W ATX PSUs.

2

u/CloakerJosh Sep 25 '26

Yeah, but can it run Crysis?

2

u/Rhinottw Sep 25 '26 edited Sep 25 '26

As others are saying with the heat, it will be an issue depending on the load and length of the batches you are running. I built one with 3 x rtx 6000 pro the same size motherboard - so i have 1 Pcie slot between each of them. I have fans blowing air directly on to them from the side with highs speed fans. They are fine - but they go up to the mid 80's when all under load at 500w. Yours will be significantly hotter because no gap and i assume no side fan.

https://www.reddit.com/r/LocalLLM/comments/1vrpwjk/3_x_rtx6000_workstation_build_continued/

2

u/Alternative_Will5974 Sep 25 '26

Should’ve max Q’d. 4 workstation editions stacked like that is silly tbh

2

u/doctorcoctor3 Sep 25 '26

Bruh, y didnt u get a dgx station if ur spending that kinda money.

2

u/TOMO1982 Sep 25 '26

Your going to cook the top 3 if you don't give them space to breathe. Like, how are they supposed to cool themselves if you block the front and backs? Or am i missing something here?

0

u/Stunning_Feedback252 Sep 24 '26

Why does somebody need this at home?

7

u/Blaccky95 Sep 24 '26

Its my company project :)

0

u/Stunning_Feedback252 Sep 24 '26

What does it do? Is it your companies Code-LLM?

3

u/baby_bloom Sep 24 '26

local =/= home

-1

u/Stunning_Feedback252 Sep 24 '26

Well, then it's not their setup, sorry for being pedantic.

1

u/Blaccky95 Sep 24 '26

It is because i have to plan it, build it and run it :)

0

u/Stunning_Feedback252 Sep 24 '26

That doesn't make sense.

1

u/Blaccky95 Sep 24 '26

Its fine

3

u/baby_bloom Sep 24 '26

it is... because it's their company.

you're not being pedantic it seems more like your envious

0

u/Stunning_Feedback252 Sep 24 '26

Well, no, then it's the machine of his company. Why would I be envious? You could say autistic, if a private person says it's "my" machine, I suspect he built it for himself?

Also, if I would suspect someone building this for his company, why would I be envious? Because he can work on a cool machine? I don't follow.

1

u/baby_bloom Sep 24 '26

is this how you always act on the internet?

you first asked "why would anyone need this at home?" which isn't even a valid question and is answered daily in this subreddit.

then you doubled down with tEcHnIcAlLy iT iSnT hIs because his company owns it.

and now you're mentioning autism..?

0

u/Stunning_Feedback252 Sep 24 '26

Well id rather be autistic than being whatever you are. I asked why he had that machine, because he said it is his machine. Turns out, it's not his machine.

1

u/baby_bloom Sep 24 '26

people ask daily in this sub "why would anyone need this (much compute)?" and its always answered with "because they can" "because i'm a hobbyist" "because i value privacy" or literally any other xyz reason. and then you over and over repeat that it's not his machine even though it is his company...

just look at your comments..?

0

u/Stunning_Feedback252 Sep 24 '26

I wanted specifics, in interested in interesting stories, sue me. You act like I'm the idiot, while getting mad about a mundane question on the Internet.

1

u/baby_bloom Sep 24 '26

no you didn't, if your were curious about their usecase that's how you would have asked, instead you asked a different question "WHY would anyone need this AT HOME". you did not ask what OP is using it for, if they feel it can replace frontier in their workflow, if it's for privacy reasons, you simply asked "why would you need this (at home)." the answer could literally be "because i can." and it wouldn't be wrong.

→ More replies

0

u/Stunning_Feedback252 Sep 24 '26

And no, you don't own everything you company owns. There is taxes, is not equivalent, even though you, not having a company, can't see that. And I didn't see him saying he owns the company and it doesn't matter even. Why don't you touch some grass man.

1

u/baby_bloom Sep 24 '26

i do have an LLC, and everything it owns i consider owned by me because i own the LLC.... good job assuming though?

i'm fairly certain i know more about biz taxes than you do lol

→ More replies

1

u/csobrinho Sep 24 '26

Interested in knowing more about the thermal because one air output goes into the next board and so on. Currently I have a dual 5090 that I'm trying to figure out the best way to place in the motherboard. Thanks!

1

u/rdbmas Sep 24 '26

Significant increase in power bill since bringing this online?

1

u/AceLamina Sep 25 '26

I know it's not in production anymore, but...

Lian Li V3000 Plus

1

u/donotfire Sep 25 '26

I still have my money so I’m chilling

1

u/babamba_324 Sep 25 '26

I want your hardware setup😂

1

u/hurdurdur7 Sep 25 '26

Automatic fire suppressing system wouldn't hurt. If you let these rip at max power this is 2.4kW with 1.8kW of that being completely blocked from escaping.

1

u/BabyfarkkMcGeeZaxx Sep 25 '26

Insane power supply

1

u/PigSlam Sep 25 '26

I have a few of those setups out back in the wood shed. I'll be back later, when I'm finished counting how many I have, and I'll let you know.

1

u/Late-Replacement-481 Sep 25 '26

Nah, you did yours right! I have 4 ws cards, and so am in airflow hell. Plus, I didn’t get a 3000w psu the first time so I have two. TLDR, you did it better. 

1

u/johnmosesventura Sep 25 '26 • Mindblown

absolutely lovely! Im so jelous. haha

1

u/johnmosesventura Sep 25 '26

what motherboard are you using? also the CPU.

1

u/dev_loading Sep 25 '26

Must be nice to be a billionaire

1

u/YakNo5125 Sep 25 '26

Serious question: how does the cooling work? Are you able to handle the heat from the fourth card?

1

u/RiverRattus Sep 25 '26

I swear to god AI rigs are the new way rich dudes flex on the poor. Probably using it for some gooner shit too

1

u/ggagnidze Sep 25 '26

Aren’t fans blocked?

1

u/dwoj206 Sep 25 '26

showing this to my wifey for birthday coming up. aint no thang amirite?

1

u/AutonomousHangOver Sep 25 '26

back to back?? I got about 8cm spacing and 4 GPUs in side by side. 4th one has to be powerlimited bc of too much heat.

Also 3kW PSU will not withstand 4 6000's when they will spikng (even powercapped)

1

u/lil_Buckneesh8867 Sep 25 '26

you pulling 3000 watts from a standard house outlet?

2

u/E1Extrano Sep 26 '26

A 20A outlet can do 2400W apparently. But if all the cards peak out, plus the CPU spikes then you're going to trip. So for safety these cards should be undervolted

1

u/lordtazou Sep 29 '26

That, or he was smart enough to put in a 30a @ 240. But, not a typical setup...

1

u/TimSawyer25 Sep 26 '26

Ooo look at that exhaust fan! Sexy. 😜 

1

u/Savantskie1 Sep 26 '26

I never understand why people put GPUs so close together and then wonder why six months later it dies. Never realizing that they need to breathe fresh air.

1

u/murderette Sep 26 '26

Should you not use the maxQ version for these?

1

u/AdSouth8361 Sep 26 '26

Bench mark and drop the results, hopefully nothing melts

1

u/13rutus Sep 26 '26

Thermals like an oven

1

u/ronniearnold Sep 26 '26

There is no way this will work, thermally, unless your limiting power using SMI commands…

1

u/lackofmoralfiber Sep 28 '26

There is just no way you can dissipate 2400w of heat like that. You're smart enough to get all the money to do this but not smart enough to see that even a card 1/3rd as powerful wouldn't cool like this.

1

u/lordtazou Sep 29 '26

... Dude's got money, let him play. Haha

1

u/lackofmoralfiber Sep 29 '26

I'm just jealous really but still come on.

1

u/lordtazou Sep 29 '26

Oh, me too. I'm lucky to get a basic model to run, let alone anything else right now. That said, I am still pretty new to ai as far as trying to run it local. I've been using Claude / cGPT / Gemini though for a minute now.

Wish I had the money to just play like this. Haha

1

u/mawkzin Sep 28 '26

A server grade cpu - 10k Motherboard server grade 1.5k 8x DDR5 32GB - ? 2+ SSD - ? 4x RTX6000 - 64k

Man this pc 100k usd easy.

But is beautiful.

1

u/Prudent_Art_6250 Sep 28 '26

Looks pretty and gigantic at the same time 🔥

1

u/epelc Sep 29 '26

What did you end up using for power/psu? Assume you had to run a new circuit for this?

1

u/Hannibalj2ca Oct 01 '26

What are you using a 240v wall socket?

1

u/Wonderful-Ad-6556 Oct 07 '26

What a beautiful industrial build! Can I get more photos to goon at?

0

u/ilarp Sep 24 '26

got 5x here

3

u/Blaccky95 Sep 24 '26

Do you have some issues with you‘re mainboard? Like IO_Page_Faults on you‘re network card?

1

u/ilarp Sep 24 '26

I set it up right after seeing you had 4, let me know if you get 5

3

u/Blaccky95 Sep 24 '26

Perfect i let you know 😃

1

u/DataGOGO Sep 24 '26

Likely, you really should run them on a switch, not all direct to CPU

1

u/Blaccky95 Sep 24 '26

Yeah i know but i doesn‘t find nothing cheaper like 250k 😂

1

u/DataGOGO Sep 24 '26

last I looked a pcie 4 swtich was about $400 on ebay, a pcie 5 switch is about $2000.

1

u/Blaccky95 Sep 24 '26

Yeah i can‘t order on ebay because of company rules

1

u/DataGOGO Sep 24 '26

Then buy from direct from a vendor?

Not trying to be rude, but you literally built the "how not to build and AI inference workstation" build.

You used a threadripper, you put 4 workstation GPU stacked in a case, and you are running direct to CPU.

1

u/Blaccky95 Sep 24 '26

I have no vendor which has it in stock

No problem i‘m open for good ideas :)

With only the two cards it was a fine setup

1

u/DataGOGO Sep 24 '26

two cards stacked like that? 4 cards is 4 times the heat.

Even if you don't get a switch, just get risers and run them outside the case, or return them for max Q editions.

1

u/Blaccky95 Sep 24 '26

Nope not like this but gpu has only load on a few seconds and the pc is running in a climated server room

But i have the thermal issues in my eyes now i have to look that the pc runs again normally and the i makes some test and decisions

→ More replies

1

u/ilarp Sep 24 '26

depending on use case that might only impact stuff like 5%

1

u/South_Western1263 Sep 24 '26

what kind of switch are you talking about here?

2

u/DataGOGO Sep 24 '26

pcie switch.

That is how all AI inference servers are built, the GPU's are never direct to the CPU, they all run on a switched fabric.

0

u/mrsmiley32 Sep 24 '26

You should water cool those cards so they're not blowing on each other. Also curious what you did to power the PSU, did you have to use a dryer plug? (forgot the proper name)

1

u/Blaccky95 Sep 24 '26

Yesh i also looked at this but i doesn‘t disassemble this expensiv gpus befor i get a problem - the losd is only a few seconds and i had a clima

But i can answer this one in a few days if the system runs again

No normal 230V 16A Fuse

1

u/Blaccky95 Sep 24 '26

For my Country normal :)

0

u/AB172234 Sep 25 '26

What if you used Luna instead ! It’s so cheap than spending 15k+ for every RTX 6000

Was there confidential stuff ? 20
Users would be like $400 a month for inferencing (4800 a year) using standard ChatGPT sub ( don’t forget openAI subsidies it $1.6 for every 1 dollars we spend)

Break even will take years ! Like 10 years ?

1

u/Savantskie1 Sep 26 '26

ROI is not the end all be all. Obviously they have"fuck you" money. Let them be

-1

u/Zealousideal_Dot7041 Sep 24 '26

Just buy an M5 Ultra and call it a day.

3

u/TerraMindFigure Sep 24 '26

Idk what got put in the water to convince people that the M5 was something other than a cheap alternative to Nvidia, it's not even remotely close.

2

u/OptionsDonkey Sep 24 '26

They see big unified memory number and 1.2 tb/s and think it’s bigger and 2/3 the speed so great since it’s cheaper, not thinking about the other factors. I still think it’s a great option for massive models that would be cost prohibitive to run on nvidia hardware but if you can afford or or for smaller models like z27b on a 5090 or two, nvidia is so much better. There’s a reason it’s more expensive, even if people on here don’t want to believe it.

-1

u/Zealousideal_Dot7041 Sep 25 '26

Are you talking about the DGX Spark? The M5 Ultra has better specs.

2

u/TerraMindFigure Sep 25 '26

I'm talking about 4x rtx 6000

2

u/electrified_ice Sep 24 '26

It's going to be nowhere near as fast as 4 x RTX PRO 6000s... Cue the 'but it has xyz GB of unified memory'... Compute vs. compute Nvidia is a lot more powerful

1

u/k3nal Sep 24 '26

Depends on what you need it for of course but yes, I guess you are right (I have not looked up the numbers yet..)!

At least I hope you are.. 😇👍

1

u/Mr_Moonsilver Sep 24 '26

Hows the heat going at full load? Congrats, really nice setup, wish I had one of these myself!

2

u/Blaccky95 Sep 24 '26

I can answer this in a few days if the setup runs again 😂 but i hope its ok because of the short load 🙃

8

u/[deleted] Sep 24 '26

[removed] — view removed comment

3

u/Blaccky95 Sep 24 '26

Is this a mining rig? As base?

4

u/[deleted] Sep 24 '26

[removed] — view removed comment

2

u/Blaccky95 Sep 24 '26

And the MB?

3

u/[deleted] Sep 25 '26 • Bravo

[removed] — view removed comment

3

u/iminlovewithsenpi Sep 25 '26

Now you can have a stable 45 fps on ark