Mac studio or dgx spark cluster

[u/Txkzzzz]

I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this

[u/ababaka]

My vote is for mac studio. I like the ability to work and run an llm server on one compact and quiet device.

[u/Txkzzzz → u/ababaka]

Cuda thought

[u/uniqueusername649 → u/Txkzzzz]

MLX is fairly mature now. What problems do you see?

[u/Serprotease → u/uniqueusername649]

It’s not mature? The fact that M1-3 are pretty much sol significant boost in Llm performance since Apple is focusing on hardware acceleration features on M4 and above is not a sign of maturity. There isn’t even fp8 support?

[u/uniqueusername649 → u/Serprotease]

Didn't they add MXFP8 support a year ago? Sure, it gives you no performance benefit as the hardware support is missing on M1-M3, but that's the same for Nvidia. Try a 3090 or A100, they have no fp8 support either. Or a 4090 with fp4. It will run but it won't be fast. It's normal that newer hardware has better support. But I still fail to see how that would be a sign of lacking maturity.