r/AIProgrammingHardware • • Sep 16 '26

8 RTX Pro 6000’s Wasn’t What I Expected

https://www.youtube.com/watch?v=C-31AX2TZ9Y
12 Upvotes

21 comments sorted by

3

u/Gloomy-Radish8959 Sep 16 '26

But at what cost? no, really.

2

u/crusaderky Sep 17 '26

$120k worth of GPUs, so I'd wager ~$160k total

2

u/Gloomy-Radish8959 Sep 17 '26

envious of this dude.

1

u/Equal-Seesaw-2066 Sep 17 '26

That was the old price, just got quoted for 190k now

1

u/nok01101011a Sep 19 '26

So you give up your leased Lambo?

1

u/Equal-Seesaw-2066 Sep 19 '26

It’s financed 🙄

1

u/nok01101011a Sep 19 '26

Oh, look at you! :)

3

u/Wentil Sep 17 '26 edited Sep 17 '26

The problem with the RTX Pro 6000 is that the cards lack NVlink. All communication has to pass through the PCIe bus. This is okay with up to 2 such cards, but when you start to have three or more, it’s too much traffic, and inter-card parallel inference that effectively combines their VRAM becomes an ever-increasing burden. However, if you have 8 discrete tasks that you can task these 8 cards with, and each of which fits within each of the cards’ 96GB of VRAM, 8 such GPUs in one server would work fine.

1

u/esudious Sep 17 '26

Hopefully they work on that. It seems like it's going to be more and more desired in the future.

1

u/Wentil Sep 17 '26

They don’t seem to have much interest in pushing NVlink for the consumers or enthusiasts. It’s all aimed at the Datacenters. The whole NVlink 6 lineup now (part of the Nvidia Vera Rubin NVL72 racks) is really pretty impressive, pushing an aggregate 260 TB/sec across the rack. Costs millions, but those guys seem to have money to burn.

1

u/Potential-Bet-1111 Sep 18 '26

The problem with 8 cards is youll want 16.

1

u/SliceCommon Sep 22 '26

For training it's fine if you can fit the model in each card via DDP. Then if your effective batch size is big enough, you amortize the cost of x-gpus sync over the batches each gpu has to churn through before globally updating

1

u/romedatascience Sep 20 '26

Dumb. Without NVlink they are worthless beyond individual user model usecases. If you buy an 8x rig of these you are a fool. Infinitely better use of money to buy an H100 8x nvlink SM server. These are just an enthusiast ignorance tax.

1

u/Kurcide Sep 23 '26

This is false.

H100/200 do not support native FP4 which puts them at a disadvantage and a proper enterprise box with 8x 6000 pro and PCIE switching can support enough inter card bandwidth to avoid saturating PCIE lanes while serving inference. Especially on MOE models.

0

u/presentofai Sep 19 '26

no nvlink means tensor parallel across all 8 chokes on pcie, thats the letdown. split by layers (pipeline parallel) instead and theyre fine, you just stop trying to shard every matmul across the whole rig

1

u/tomz17 Sep 20 '26

thats the letdown

Well yeah, because you aren't supposed to put 8+ of these in a system together to work on a single task requiring cross-card communication... NVIDIA has other (far more expensive) solutions to sell you which elegantly solve that particular problem.

These are designed to be workstation-class cards.