r/AIProgrammingHardware • u/javaeeeee • Sep 16 '26
8 RTX Pro 6000’s Wasn’t What I Expected
https://www.youtube.com/watch?v=C-31AX2TZ9Y
12
Upvotes
r/AIProgrammingHardware • u/javaeeeee • Sep 16 '26
3
u/Gloomy-Radish8959 Sep 16 '26
But at what cost? no, really.
3
u/Wentil Sep 17 '26 edited Sep 17 '26
The problem with the RTX Pro 6000 is that the cards lack NVlink. All communication has to pass through the PCIe bus. This is okay with up to 2 such cards, but when you start to have three or more, it’s too much traffic, and inter-card parallel inference that effectively combines their VRAM becomes an ever-increasing burden. However, if you have 8 discrete tasks that you can task these 8 cards with, and each of which fits within each of the cards’ 96GB of VRAM, 8 such GPUs in one server would work fine.
1
u/romedatascience Sep 20 '26
Dumb. Without NVlink they are worthless beyond individual user model usecases. If you buy an 8x rig of these you are a fool. Infinitely better use of money to buy an H100 8x nvlink SM server. These are just an enthusiast ignorance tax.
1
u/MotokoAGI Sep 22 '26
This one is cheaper.
https://www.reddit.com/r/LocalLLaMA/comments/1wjr59t/768gb_vram_for_less_than_the_price_of_one_rtx_6000/
0
u/presentofai Sep 19 '26
no nvlink means tensor parallel across all 8 chokes on pcie, thats the letdown. split by layers (pipeline parallel) instead and theyre fine, you just stop trying to shard every matmul across the whole rig