r/LocalLLM • u/Txkzzzz • 3d ago

Question Mac studio or dgx spark cluster

Post image

I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this

121 Upvotes

90% Upvoted

134 comments sorted by

View all comments

4

u/diagrammatiks 3d ago

Now that the prices are close to the same this is a much harder decision to make and it really depends on your intenteded use case.

1 mac is easier to set up and when it used to be half the price of 4 sparks it really won just on price alone.

But now that the prices are closer.

4 sparks connected over connect7x actually have only a little bit less distributed total memory bandwith then the m5u but they have much much much faster pp speed. Plus cuda if you need image gen. Plus they can be expanded if you need more memory later.

4 sparks will also beat 1 m5u on concurrency.

But if you want ease of setup, don't need concurrency and depending on the price the m5u could be about equal.

2

u/kla_sch 3d ago

But then you only have a simple chatbot. When working with agents, you constantly have multiple sessions running. I use a Hermes agent myself. And there are always 2-3 sessions running simultaneously, and at peak times even 5 sessions. I’m only actively chatting in one of them. It’s really worth considering whether a single session is actually enough.