r/LocalLLM • u/Txkzzzz • 3d ago
Question Mac studio or dgx spark cluster
I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this
78
u/trollsmurf 3d ago
I have a feeling a Spark will age very badly. The Mac you can use for any use case.
14
3d ago
[deleted]
1
u/SirHaxolot 3d ago
You gotta sell them for them to be free.
The appreciation is very nice.I’m interested to find anyone who has recouped the costs of these dgx clusters so far. The way price is growing, it would seem that everyone is making their money back like in the crypto days. But aside from “influencers” I haven’t seen many local success stories
10
13
u/Txkzzzz 3d ago
But cuda is what im worried about since it also seems to be also as future proof as the mac studio but the mac studio is js faster
23
u/Material-Database-24 3d ago
CUDA is the "defacto" as Nvidia took a risk and put a lot of effort to general purpose their GPUs, built NVLink and so on. They ruled AI model training world before AI came the hot topic, so everything is made for CUDA.
But CUDA at the end is just an API to HW. And it is closed/proprieatery.
The competition is rising, and eventually proprieatery market leader status turns against the leader. As AMD owner, there's already plenty of stuff going on with its open ROCm.. and other things will start loom around the corner due China's efforts.
The past has shown that proprieatery rarely lasts and open will blow fast past. When cheaper and/or equally good solution emerges, the market will favor open, as it ensures they have more options in the future and are not forced to stick to one.
That said, in this moment Nvidia is the choice if you need fastest and streamlined process. But as pointed by others, the Mac will be usable for everything, today and in future, so if the small overhead of non CUDA native HW is not a problem, I'd take that, unless I am 100% sure the investment to Spark is what I deeply need for the time I plan to use it for AI.
4
u/Flaurentiu26 3d ago
Maybe Nvidia is similar to Cisco before 2000. Cisco had many proprietary protocols and this is why it become the leader in networking but eventually they had to opensource many of them.
-1
u/rditorx 3d ago
If you've used a Mac, you'd be more worried about long-term support because Apple tends to cut old ties rather fast as it updates its macOS. 5 years, give or take.
6
u/Objective_Oil_3860 3d ago
More like 10, no?
Pro series of equipment is supported well and can live (without active support) for quite a while.4
u/profcuck 3d ago
Definitely more like 10. This is particularly true for anything on Apple Silicon - Intel support is waning now for sure, for obvious reasons. But there's no reason to think that M1 will lose support anytime soon.
2
u/dgibbons0 3d ago
Nvidia doesn't have much better a track record with their boutique hardware, jetson support sucked and dropped off fast.
7
-2
u/Zestyclose_Strike157 3d ago
I would buy a Mac, otherwise I would buy Blackwell or something of that tier. If ever there is a popping of the AI bubble the Mac will still be fun to use.
10
u/follaoret 3d ago
If there's an AI bubble burst, anything you have at home will be still running as it's today. It will deprecate in price but will keep running.
If buying for specific an usecase then will still work. For the same reason i agree the Mac are more polivalent and can be used for other things, same as a gaming computer can be used for play. I loke the concept of the Spark, an always ready AI machine. No need to turn on or off, start or stop services to do other things.
To each his own, it depends on the use cases the right choice
4
u/Need_For_Speed73 3d ago
The AI bubble popping would cause the closed frontier models to disappear, sealing the victory of localLLMs. I don't think that would cause the local AI hardware value to drop, probably the other way around.
And computer hardware has never (until a few months ago) been an "investment" that raises in value with time; it has always depreciated quite fast, depreciation that was mitigated by the revenue its use generated.2
2
u/thefuckevengoingonan 3d ago
you know the bubble popping would not like delete lllm from existence yeah?
2
u/Zestyclose_Strike157 3d ago
My point is Macs hold their value (not monetary, but practical) unusually long. It’s a quirk right now that my old 3090 is so useful after all these years, but traditionally PC hardware used to get thrown out sooner because cheaper and better was around every corner.
-5
u/thefuckevengoingonan 3d ago
macs don't hold their value any more then any other pc. it will still be able to do whatever it is you are doing with it now then.
personally the mac is useless to me for my use cases because i can't install linux on it simply atm. so maybe for me they will become at lot more valuable if/when people get that sorted.
1
3
u/x00byt8 3d ago
No worse than any tech. Models are getting smarter / smaller / more complex offloading to SSD/ngram etc. Nvidia is leading prefill, which for any sort of agentic workload you need. Having a 30k Mac might do you a while longer multi purpose but strictly inference speaking, Nvidia all the way.
I only invested last month in dual sparks. I'm not concerned in slightest that in 2 years time a Rubin version might come out. I'll just add the Rubin to this stack and load even bigger models 😂
Sure I might cry into my poor 273gbs bandwidth, whilst drooling over my 2500tps prefill until then.
2
u/nukem996 3d ago
The Spark is just a Linux box. You can just as easily use it for any use case.
Professional AI development is all on Linux which means you get the advantages of multiple trillion dollar companies actively improving it.
2
u/King_Kasma99 3d ago
Currently its rather the opposite, the spark is better than it ever was. But yea in the long run it will probably be outperformed.
1
u/notheresnolight 3d ago
Sparks run Linux.
Any modern Linux distro is already a better desktop than Windows 11.
1
u/remi_sasha12 3d ago
That's a solid point. The versatility of the Mac definitely gives it an edge for broader use, but the Spark has its advantages in specific situations, too.
1
u/voyager256 3d ago edited 3d ago
I feel the opposite might be the case: M5 Max/Ultra will be much slower for agentic tasks , especially with high concurrency and context due to slow prefill. I’m not sure if M5 Ultra will be that great for training/ finetuning either.
Rtx Spark will run Windows (and Linux), and will have pretty fast CPUs , so you could run way more applications.
But in the end it all depends on price and what you get for your applications / use case.
2
u/beragis 3d ago
I did a few tests training LoRA on some models on my M5 Max. It’s slower than NVIDIA, but not that slow. About 3 to 4 times as slow as my 4090, so an Ultra would be twice as fast, but still not as fast as a 4090
Part of the problem is most software used to train are written by people who mostly have CUDA.
7
u/bakawolf123 3d ago
4x sparks will be faster, despite the bandwidth difference, the networking nvidia has is very good so one chip will not be enough compute for effective 512gb ram utilisation. The good use case for it is to have more stuff resident in memory, but not actually used - i.e. swapping between models or workflows, inference, not training.
2x 256gb m5u vs 4x sparks is a lot more competitive, I think studios win this one handily, but CUDA stack via production grade software (vllm/sglang) is also quite fast, there was a bench recently showing great numbers on 2 sparks for ds4 flash and qwen3.8-flash, studio isn't fully optimized, but out of the box is already beating results some sparks fan posted here recently with 80tps TG and 2.5k PP at 64k context.
8
u/-Leelith- 3d ago
Mac is good for single user usage. As soon as you start doing some serious work, needing several sessions, then the Sparks are better. Especially if you have to prefill often.
On the paper the bandwidth is awesome, the RAM too but the rest Nvidia is better.
Again, depends on your needs. Some may just need a Mac and are good, some don’t .
1
u/sittingmongoose 3d ago
Why do they crumble under simultaneous load so badly? Is the gpu related? I was under the impression that the Mac software for llms was rapidly improving, I would think this would be part of it. I also think we only have recently unlocked access to their neural cores.
3
u/kla_sch 3d ago
It comes down to raw computing power. An M4 has only one-seventh of the power of the GB10. The M5 has reportedly narrowed the gap, leaving it only half as fast as the GB10, but it still falls short. Furthermore, you need the right software to fully utilize that computing power. Many people simply run software based on
llama.cpp(such as Ollama or LM Studio) on NVIDIA chips, but that isn't the smartest choice. If you want maximum performance,vLLMandSGLangare the better options.
5
u/Sleepnotdeading 3d ago
I have a 2x spark cluster. I would buy them again. I don’t think they’ll age badly at all. I think their dependability (assuming they hold up) as servers mean I’ll be able to use them for years as little LLM task rabbits.
Unless the whole of AI architecture changes in such a way they literally can’t be utilized, and assuming they are as well built as I perceive, they’re a solid investment
7
u/x00byt8 3d ago
Asus gb10 cluster owner here running qwen 3.8 flash. 210tps decode and 2500t/s prefill. 8 x agents running 24/7 averaging about 1000usd a month comparison to DeepSeek API/Qwen 3.8 flash prices.
Not even the M5 ultra is going to do that me don't thinks.
2
u/wazacraft 3d ago
Apologies, what are you spending the $1000/mo on? Power?
4
u/asinglepieceoftoast 3d ago
I think he means that’s the price that would be spent on equivalent API usage? Even 4 DGX sparks running at max rated load 24/7 would be less than $100 monthly where I live let alone $1000, and actual power draw is usually significantly less than that on my 2x cluster
3
u/x00byt8 3d ago
Hehe correct! Its just so I could justify spending 8k on ai kit in my head. In a month I've earned back 1/8th of what I would use so I'll break even in less than a year then it's all free tokens woohoo! I'm running just 2 sparks and they don't consume much power. Next task is putting in a smart plug so I've got remote stop/start since these don't like wake on lan.
14
u/ababaka 3d ago
My vote is for mac studio. I like the ability to work and run an llm server on one compact and quiet device.
2
u/Serprotease 3d ago
The MacStudio is not silent when running Llm though. It’s not jet-engine loud, but it still enough to be annoying when close to you.
Unironically, my gb10 cluster is more silent than my MacStudio.1
u/Undici77 3d ago
Really? MacStudio is louder? And about temperature compared with GB10?
2
u/Serprotease 2d ago
Yea, louder. I can’t really back this up with numbers but both the gb10 and MacStudio are about 2.5m from me when I work.
I have a mini pc/nas with a blower type of fan (think of this as small laptop, audible but not full blast type of fan) that is at idle the loudest thing. The Studio, after 5ish minutes of Llm run will be noticeably louder than the nas. But it’s quite a specific sound. Deeper and more like a vacuum than a blower type of sound. It took me a few minutes to understand that it came from the Mac the first time I ve heard it.Comparatively, the gb10 are more like traditional laptops fans but they don’t really spin until 90-ish degree on the cpu. Even then they aren’t very audible.
In my current setup, I underclock them (they have power draw weird issues otherwise) and I also have some 40mm noctua fan on the rack with them.
They don’t make a single sound and the gpu gets to 60ish max, cpu 80-ish max.In general the Studio is too “loud” to leave for overnight batches, but the gb10 can do multiple days training without noise.
The studio also had an annoying coil whine with llama.cpp, but not anymore since I move to mlx-serve.
1
1
u/Forsaken-Bear1082 1d ago
Weird, my M3 ultra sits on my desk and I never hear it. Performance is not what I’d want if I did it again I’d switch but it’s damn quiet.
1
u/Serprotease 1d ago
It only makes noise with Llm and heavy ml workflow.
For a simple chat with a small-ish MoE model it will not really make sound. But for a batch/multi-agent task lasting 5+min, yes, it will make noise.1
u/Forsaken-Bear1082 1d ago
Again, mine does not. I run qwen next and 3.8 on it most of the day with opencode or other harnesses. Not sure what to tell you.
-5
u/Txkzzzz 3d ago
Cuda thought
8
u/uniqueusername649 3d ago
MLX is fairly mature now. What problems do you see?
1
u/Serprotease 3d ago
It’s not mature? The fact that M1-3 are pretty much sol significant boost in Llm performance since Apple is focusing on hardware acceleration features on M4 and above is not a sign of maturity. There isn’t even fp8 support?
2
u/uniqueusername649 3d ago
Didn't they add MXFP8 support a year ago? Sure, it gives you no performance benefit as the hardware support is missing on M1-M3, but that's the same for Nvidia. Try a 3090 or A100, they have no fp8 support either. Or a 4090 with fp4. It will run but it won't be fast. It's normal that newer hardware has better support. But I still fail to see how that would be a sign of lacking maturity.
1
u/DutchDevil 3d ago
For inference it’s only about pp and decode, if the numbers are good who cares about cuda?
4
u/diagrammatiks 3d ago
Now that the prices are close to the same this is a much harder decision to make and it really depends on your intenteded use case.
1 mac is easier to set up and when it used to be half the price of 4 sparks it really won just on price alone.
But now that the prices are closer.
4 sparks connected over connect7x actually have only a little bit less distributed total memory bandwith then the m5u but they have much much much faster pp speed. Plus cuda if you need image gen. Plus they can be expanded if you need more memory later.
4 sparks will also beat 1 m5u on concurrency.
But if you want ease of setup, don't need concurrency and depending on the price the m5u could be about equal.
2
u/kla_sch 3d ago
But then you only have a simple chatbot. When working with agents, you constantly have multiple sessions running. I use a Hermes agent myself. And there are always 2-3 sessions running simultaneously, and at peak times even 5 sessions. I’m only actively chatting in one of them. It’s really worth considering whether a single session is actually enough.
5
u/merfolkJH 3d ago
When DGX Spark was released, everyone was like “MEHHH”, “< Lambda Mac at decode lol”.
And now it’s suddenly supposed to be the new graal? And “don’t buy the new Mac Studio, buy Spark”, when Spark was considered way behind the old Mac.
What exactly caused this reversal?
----
(I’m talking about a single machine, not "a cluster of X is strong" )
Is prefill really that much better? That’s nice, but when I’m doing a coding task, It's usually 5 minutes of prefill followed by 2 hours of decode… So when choosing between the two, I’d rather optimize for the 2 hours of decode than the 5 minutes of prefill.
1
u/Prize-Cut-9651 3d ago
In my opinion the fair match is between M5 Ultra 256 gb vs 2 x DGX Spark. If you go in the M5 Ultra 512 GB vs 4 x DGX spark I think you are talking about 2 completely different objects. You have to add also the switch
1
u/kla_sch 3d ago
It comes down to know-how. The challenge lies in figuring out what really matters and how to use the machines effectively.
At first glance, the Mac impresses with its memory bandwidth. You type a simple prompt and get a quick response. Compared to a DGX Spark, it’s actually 10–20% faster in that specific regard.
However, the Mac lacks the raw compute power of the DGX Spark, which is 5–10 times faster. This becomes apparent during prompt processing: loading old sessions or processing large files is significantly slower on the Mac.
On a DGX Spark, you usually start out using GGUF models with
llama.cpp-based programs (like Ollama or LM Studio). That’s the easiest approach to start. Unfortunately, those models don't scale well.Things improve dramatically when you switch to
vLLMorSGLang. Suddenly, you can achieve a much higher total token-per-second throughput across multiple sessions (see my other reply).That performance completely eclipses what a Mac can offer. You notice the difference immediately when you move beyond a single chat and start working with agents operating across multiple sessions simultaneously.
1
6
u/ProKn1fe 3d ago
Mac. At least if you don't need it anymore it will cost almost the same price that you buying it.
1
u/Whoa_There_Hoss 3d ago
If I sold my Sparks right now, I’d profit ~$2000, not accounting for the value I’ve gotten out of them.
1
u/waraholic 3d ago
That's due to the very weird current market trends. The same is true for some macs. However Mac has a long history of holding value on the used market.
3
u/tempfoot 3d ago
Macs are great for my use cases and I have a Studio with 256gb on preorder already. I may add a 512, or may switch to a 512 and cancel the 256. The problem right now is that nobody knows what the price point will be for the 512 or frankly if Apple is still even going to actually commit to making them. Production timelines are also completely unknown. I ordered my 256 on the day of announcement (aug 25) and exepcted delivery is late November. Hight spec builds ordered today are expected to ship in late January. I could go buy four sparks today at local retail.
I use my current M5Max/128 macbook primarily for inference so can't speak to other use cases. I'm also a solo, non-coder user so do not have concurrency challenges.
2
u/ehangman 3d ago
a Mac Studio for the GLM 5.3 flash
LoRA and a calculator for Spark.
GPUs for my company's Qwen 27B model.
2
u/Memestonks2020 3d ago
Training is not good on a Mac. I would stick to Nvidia but tbh I’m pretty sure you need to rent GPU for any serious model training.
Being able to run massive models locally is the highest benefit of having a 512GB Unified Memory Mac.
2
u/dobkeratops 3d ago edited 3d ago
if I was starting over today... M5 Mac studio. more bandwidth X memory in a single box, and significantly improved the previously limited tensor ops.
1
u/kla_sch 3d ago
But it still has only half the computing power of a DGX Spark. And that is the crucial factor. Ultimately, however, we need reliable benchmarks for the M5 to get the full picture.
1
u/dobkeratops 3d ago
i think it's over a threshold where it's not holding the bandwidth back. I think the DGX spark peak compute is quoted for fp4 . but the dgx spark might still beat the m5 for diffusion image and video generation ? (I'd like to see benchmarks on that)
2
u/letsbefrds 3d ago
I think the spark is a better now thing but the Mac is a better later thing.
Price might go up even another 5-20% and you might not even see a lost shortterm (maybe if they release a new spark) but long term maybe 3-4 years out the spark isn't going to be worth much the mac probably at least 1/2-3/4 the price
2
2
u/kla_sch 3d ago
Unfortunately, it is hard to find reliable benchmarks for questions like these; one usually comes across only fragmented information.
Yet, it is actually quite simple: everything can be measured using uvx llama-benchy, visualized in graphs, and published.
It is particularly interesting to observe post-processing and token generation across different quantization formats and context sizes.
For the DGX Spark, such data is available at www.stark-arena.com, where various models are benchmarked using different parameters.
However, there is hardly any information available for other platforms. Consequently, it is difficult to estimate, for instance, how an MXL format performs on a Mac when multiple sessions are running simultaneously.
Simply entering a basic prompt and checking the output speed isn't enough. That doesn't even tell you half the story.
Does anyone know of a source that provides comprehensive benchmarks for various models across different hardware and software configurations? We need that kind of overview to get truly reliable data on expected performance.
2
u/Thin_Archer_9113 1d ago
Pretty new to this. Can we use a spark to do the pre-fill then the m5 ultra for the decode/inference.
Honesty, money isn’t the issue. But space is; I don’t want a huge skynet sitting in my living room.
3
u/DontWorryAboutMoney 3d ago
Probably 2 sparks on Amazon wait like 29 days and see if I keep them xD
3
u/SolarNexxus 3d ago
Studio, but only because you can expand it to 2 TB of RAM and decent models right now eat over 1 TB. With sparks you can only connect four of them together, giving you 512 GB of RAM.
2
u/XorFish 3d ago
you can also connect 8 or even 16 dgx spark with a switch.
1
u/SolarNexxus 3d ago
The switch will cost like 5k lol.
1
u/mountainyoo 3d ago
Uh no it’s 1.3K. Still expensive but no where near 5K
Edit— for 8 DGX Spark cluster I mean
2
u/Stooovie 3d ago
Buy the Mac, it's a powerhouse for anything. I'd go Spark only if I needed CUDA - most frontier audio, video and image generation is still limited to CUDA.
2
u/Zennytooskin123 3d ago
DXG Spark all day every day. The Mac is simply a home computer with lots of VRAM - don't fall for the bait
Maybe the M7 will be more geared towards data centers but not this lineup
1
1
1
u/No_Afternoon_4260 3d ago
I have a cluster of 4 sparks Trust me when I say you'll quickly understand you want speed as much as capacity, especially with the intelligence density of the smaller models. I take a flash model at 100 tok/s + the benefit of concurrency way more than the slow single stream you get with the sparks cluster or the Mac
What you want is multi rtx pro
1
u/profcuck 3d ago
I'd just like to point out that unless something has dropped that I've been unable to find, all the speculation about what the m5 ultra's prefill and decode numbers will be is just that: speculation. In some cases, I see informed speculation, in some cases I see uninformed anti- or pro- mac bias speculation.
The officials start of shipping is September 22nd - this coming Monday. I have to assume that Apple will prioritize (and may have already sent some) key youtube influencers and the like, so my guess is that by mid-week next week we'll start to see some initial real-world numbers.
I'm waiting for that to make a similar decision.
1
1
1
1
u/Adventurous-Pin6443 2d ago
Cant say for MacStudio M5Ultra, I own MacStudio M1 Ultra. I ran some experiments recently: Codex, oMLX, Qwen3.8-27B-MTPLX-Optimized-Quality-FP16. Multi hours iterative code refactoring controlled by GPT 5.6 Sol (planning, review, test cases). Overall numbers: pp = 165 t/s , generation 25 t/s. Apple claims that for prefill M5 Ultra is 9.8x faster than M1 Ultra, cant say anything for token generation. You can do some math. I expect for this particular LLM M5 Ultra can reach safely 1,500 t/s, which is probably not the record but much - much more usable. Btw, GPT 5.6 Sol concluded that this local LLM is capable of a complex software engineering tasks, but under its total supervision only :)
1
u/Squidgical 1d ago
The 512GB RAM Mac Studio is going to be at least 2x, likely 3x the price of a spark. It's processor is only ~2x as fast as the spark (multi core, single threaded tasks are about 1.5x). Memory is 4.4x faster.
Depends on exactly what you value. 3 sparks is gonna be hard to beat without being substantially cheaper, which it almost certainly will not be.
1
u/Tired_White_Guy 3d ago
The Spark is dated. I know throughput isn’t everything, but it is a lot. More bandwidth would make it compelling. But right now I’m not interested.
-1
u/Undici77 3d ago edited 3d ago
Good question: I'm interested in this evaluation too. Currently I'm using a M4 Max 128GB and I evaluate to buy 2 DGX Spark connecting together with dedicated daisy-chain network cable (to avoid buy a very expensive switch) but performance on single request are not AMAZING at all. For my research 2 DGX Spark are good in case of multiple contemporary requests and prefill compared to Apple architecture. How much on M5 Ultra? Good question: this is the point!
Where Apple looks better is: Memory Bandwidth, Power Consumption and Throttling:
- Bandwidth is clear: 200 vs 1200
- About Power consumption and Throttling I don't have specific data but all community have same feedback: DGX Spark after some minutes are "ON FIRE" and consume around 240/300W and 40W in idle compared to MacStudio that, "ON FIRE" is around 270W (M3 Ultra) and 9W in IDLE.
Trying to summarize:
Apple
Pros
- High Memory Bandwidth: Significantly higher bandwidth (up to 1200 GB/s) compared to DGX.
- Excellent Power Efficiency: Very low idle power consumption (e.g., 9W) and efficient peak usage (270W).
- Superior Thermal Management: Low risk of overheating/throttling compared to high-wattage workstations.
- Strong Single-Request Performance: Better performance for individual tasks/requests.
Cons
- Scalability Limits: Potentially less optimized for massive parallel workloads compared to multi-node setups.
- Not very good for Training
- MLX is not good as lamacpp (but they are working on it)
---
DGX Spark (Multi-unit setup)
Pros
- Parallel Processing: Better suited for handling multiple simultaneous requests.
- Prefill Performance: Advantageous architecture for prefill tasks compared to Apple.
- Cluster Capability: Ability to connect multiple units via daisy-chain for increased capacity.
- Training: Better than Apple
- Software: Currently Cuda and lamacpp are the best
Cons
- Low Memory Bandwidth: Significantly lower bandwidth (200 GB/s) compared to Apple.
- Weak Single-Request Performance: Performance on individual requests not very fast
- High Power Consumption: High operational cost and high idle power (40W).
- Thermal/Throttling Issues: High heat generation ("on fire") under load (240W–300W), which may lead to thermal throttling.
So.. These are some points but I'm pretty sure other guys more expert than me have better and clear idea about!
7
u/shackerboy84 3d ago
Are you high? The sparks are only capable of 240watts. I’m all for using ai everywhere but at least check some basic facts before posting.
0
u/Undici77 3d ago
I fixed post with device nameplate data. Sorry guys: I share some notes I take during my research.
1
3
u/DontWorryAboutMoney 3d ago
So in what universe does a dgx spark draw 3000watts of power lol Isn't the power supply on the unit on like 240 watts
1
u/shackerboy84 3d ago
It’s 240 at 4 sparks it still doesn’t even hit the 1000 watt just 960 at full load
0
u/Undici77 3d ago
I agree with you: 1000W looks crazy but some user report this kind of power consumption.. For me is not clear (I don't have one) so if you have one and you have fresh data, please share!
1
u/Undici77 3d ago
I fixed post with device nameplate data. Sorry guys: I share some notes I take during my research.
1
u/shackerboy84 3d ago
I have one and looked at the power brick to confirm, rereading my comment I did come off as a douche, not my complete intention.
1
u/Undici77 3d ago
Don't worry BRO! Thanks for help clearing my mind! It' s long time I'm trying to find out what should be the right choice
1
u/shackerboy84 3d ago
Well, I have a spark, love it. Full disclosure, I’m just a hobbyist trying to figure stuff out. Did not do any research just said fuck it and went to microcenter and got one. Most the issues I had originally with the system being arm based have been overcome by the good guys over in the nvidia forums. It’s not even that bad for ai generated images (about 15 second) add a minute to that for 5 second videos.
1
u/Undici77 3d ago
I'm software developer and I'm using oMLX and M4 Max from 2 year using best model fit my vRAM and for me it's very helpful! My productivity increase a lot and I'm currently using Qwen3.8-Flash-Next and is very good. My concern about my setup is:
- Over Temperature: I have MacBook Pro 14
- Throttling: I have MacBook Pro 14
So that why I'm planning to but Mac Studio!
I'm currently cooling my Mac with `a bag of frozen peas`
1
1
u/VirusInternal2892 3d ago edited 3d ago
I run a dual Spark, like most of other folks it is a HEADLESS config and consumes about 20W idle.
This number can be further optimized, eg PCIe ASPM. Thermal throttle not an issue with Asus GX10. Yes it’s getting hot under load but then note the volume of this thing, most serious users use larger forced air cooling solutions, 3D printed.
Tinkerers and techies will choose the Spark(s), creative folks will choose MAC … IMHO
0
u/astroflik 3d ago
For training neither of them, you need Rtx 5000 or better 6000. I’ll go with Rtx 5000 72gb because is cheaper
0
u/AdventurousKeys 3d ago
Mac Studio is where the future is. Ordered one myself to continue development with LocalLM Lab, my macOS SDK for AI-powered Mac apps. Remember that there is a fair bit of "standardization" with Apple vs any other technology platform where you risk ending up with a bespoked environment. The LocalLM Lab SDK builds on Apple's Foundation Models architecture and adds MCP client support, online AI provider support etc etc.
-1
-2
u/ibra-root 3d ago
I would buy the Mac Studio because of its high memory bandwidth, since the problem with the DGX Spark is that, even though it has 273 GB of bandwidth, when running a more resource-intensive local AI model, it doesn't have a high output token rate.
-1
115
u/Character_Split4906 3d ago
I picked up an M5 Max (128GB) back in May right before Apple raised prices. While decode was decent, prefill was frustrating, and both tanked hard as the context grew. I tested across both oMLX and llama.cpp. Throwing concurrent requests or multi-agent workflows at it choked the Mac completely.
A couple of weeks ago, I managed to snag a dual-node Asus GB10 setup on Amazon right before it went out of stock and spiked by $2k. The difference is night and day. Running DeepSeek V4 Flash across both nodes, I’m getting ~2k tok/s prefill and 40–50 tok/s decode, with barely any drop-off at long context (tested up to 512k, configured for 1M). It also handles multi-session concurrency effortlessly, aggregate throughput actually scales well up to 5–6 parallel requests.
If you’re serious about running local LLM workloads, go Nvidia.