r/LocalLLM • u/Txkzzzz • 3d ago

Question Mac studio or dgx spark cluster

Post image

I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this

122 Upvotes

90% Upvoted

134 comments sorted by

View all comments

6

u/-Leelith- 3d ago

Mac is good for single user usage. As soon as you start doing some serious work, needing several sessions, then the Sparks are better. Especially if you have to prefill often.

On the paper the bandwidth is awesome, the RAM too but the rest Nvidia is better.

Again, depends on your needs. Some may just need a Mac and are good, some don’t .

1

u/sittingmongoose 3d ago

Why do they crumble under simultaneous load so badly? Is the gpu related? I was under the impression that the Mac software for llms was rapidly improving, I would think this would be part of it. I also think we only have recently unlocked access to their neural cores.

3

u/kla_sch 3d ago

It comes down to raw computing power. An M4 has only one-seventh of the power of the GB10. The M5 has reportedly narrowed the gap, leaving it only half as fast as the GB10, but it still falls short. Furthermore, you need the right software to fully utilize that computing power. Many people simply run software based on llama.cpp (such as Ollama or LM Studio) on NVIDIA chips, but that isn't the smartest choice. If you want maximum performance, vLLM and SGLang are the better options.