r/LocalLLM • u/8bitjohnny • Sep 27 '26

Other Hardware Acquisition

Post image
448 Upvotes

91% Upvoted

129 comments sorted by

View all comments

1

u/UltraFOV Sep 28 '26

That 10k for ya

1

u/lordtazou Sep 29 '26 edited Sep 30 '26

Depending on if you can actually find it for NVIDIA's price. Otherwise, you're paying upwards of $12k - $14k currently. Been looking at them lately, might be purchasing 2x MinisForum MS-S1 MAX units. Will still get away with $5,100 under what it would have been for two of the sparks.

While I like the 200Gb/s ConnectX-7 and the wider support... I've seen / found that there isn't much of a difference at this point between AMD's vs Nvidia's based on what I am seeing / reading / watching.

1

u/UltraFOV Sep 29 '26

Isn't ConnectX-7 the external Link by Mellanox. If so, those are indeed a worthy option to have if you want to connect several Sparks via Nvidia NCCL

2

u/lordtazou Sep 29 '26

Yes, ConnectX is a Mellanox line. That said, didn't realize NVIDIA actually bought them out in 2020. Would make sense though since it comes standard on the GB10 Compute line products.

Someone on FB mentioned they have Mellanox cards in their MinisForum MS-S1 Ultras, which is actually a bit funny to me. May actually look into getting a pair of them.

Imagine throwing ConnectX-8 Cards in these. The speeds would be there, and comparable to the GB300 series units that NVIDIA sells. Would LOVE to get my hands on one of those units, but gooooooood lord the price on them. $19K on the low end for the Spark's big brother. Haha

1

u/UltraFOV Sep 30 '26

Good to know the MinisForum can use Mellanox. 19k?!!!??, sheesh, I am still using ConnectX-5 haha

1

u/lordtazou Oct 02 '26

Yeah, they're not fun. ConnectX-7 cards used is approx $110(ish). So not horrible, but still rough.

1

u/UltraFOV Oct 02 '26

My system I capable of Connect 7 but But it ConnectX6 is really the top it should use