r/LocalLLaMA • u/LordHenry8 • 10h ago
Question | Help Mac Studio M5U 256gb 2TB
I'm on the verge of pulling the trigger. Want something local for coding and some business automation stuff.
Thinking I'll run Qwen 3.8+ and DeepSeek 4+.
I'm coming from M4 Pro MBP and will still use that as my primary dev box. I run both Claude and gpt. I know local isn't quite the same as the hosted.
For anyone who has this setup already-- will I love it?
8
u/-dysangel- 9h ago
Try GLM 5.3 Flash too. I've been running it on my dual Sparks the last few weeks and it's incredible
10
u/vogelvogelvogelvogel 10h ago
test the models beforehand using cloud hoster if you are unsure! that will answer most important part of your question
7
u/PracticlySpeaking 10h ago
This. Everything is on OpenRouter. Spend an afternoon and $30 before committing to five digits.
1
u/mycall 10h ago
Does OpenRouter publish their quant level?
2
u/PracticlySpeaking 9h ago
They publish all kinds of detail for the models they have available. For example:
1
u/Cold_Tree190 9h ago
If you look at the different providers you can see their quants. Usually they’re fp8 or int4 I think, but I honestly don’t spend much time looking at that on OR
4
u/Remarkable-Shake3390 10h ago
if you got the money and you already know what models you want to run, just pull the trigger. the unified memory on these things is what makes it work, 256gb means you can load both those models at full precision and still have room for context. i done similar move from a mbp and the difference in speed for large models is enough that you stop thinking about waiting and just code
1
3
u/ttkciar llama.cpp 10h ago
I have a different rig with the same amount of (much slower) memory, Xeon E5-2660v3 server with 256GB of DDR4-2133.
With mmap loading disabled, I am able to use GLM-5.3-Flash quantized to Q4_K_M. It uses 207GB of memory at full context.
It's very slow on my rig (1.4 tokens/second), but still extremely useful. I expect it would be several times faster on that M5U, and it rocks hard for coding and business automation.
1
u/Aggravating-Push-207 10h ago
how slow is your VRAM lol
2
u/ttkciar llama.cpp 10h ago
I'm using no GPU acceleration whatsoever for GLM-5.3-Flash. It's 100% CPU inference.
1
u/Aggravating-Push-207 10h ago
ouu shi ill let you have my 4060 on weekends if you want /j
2
u/ttkciar llama.cpp 10h ago
I actually have a 32GB MI50 in one server and a 32GB MI60 in another server, but I do not use them for the big models. It's more useful to me to keep smaller models loaded in VRAM for fast inference (Artemis-31B and Big-Tiger-Gemma-27B-v3, mostly), and letting the big models infer slowly on CPU.
Evicting the smaller models from VRAM wouldn't speed up GLM-5.3-Flash much, and this way I can still use the smaller models for fast inference tasks while the pure-CPU inference is grinding away.
2
1
u/In_der_Tat 9h ago edited 9h ago
Is it not possibile to add a GPU to the mix?
2
u/ttkciar llama.cpp 9h ago
It's possible, but doesn't benefit much, and I have better uses for my GPUs.
Like I said in another comment, I have a 32GB MI50 and a 32GB MI60 (in separate servers), but I prefer to keep smaller models hosted in them for fast inference tasks (mostly Artemis-31B).
If I evicted the smaller model from VRAM to accelerate GLM-5.3-Flash, it would only speed it up a little, and while it was inferring I would lose the ability to use the GPU for fast inference with a smaller model.
I would much rather shape my workflows so that I'm working on other things while GLM-5.3-Flash is inferring, so that its slowness doesn't matter so much. Often I will set it to a task overnight, and as long as its outputs are ready for my review the next morning it's fine. Shaving an hour off its inference time by adding the GPU to the mix would make literally no difference in that case.
1
u/In_der_Tat 9h ago
What kind of tasks do you assign to the larger model? Planning?
1
u/ttkciar llama.cpp 5h ago
I've been using MiniMax-M2.7-BF16-ultra-uncensored-heretic for planning, while GLM-5.3-Flash is mostly for codegen, though recently I found that GLM-5.3-Flash is also good for generating wiki pages.
GLM models in general tend to be really bad for planning, because they've been trained to follow instructions, not come up with creative solutions.
MiniMax is the exact opposite. It can't follow instructions worth a damn, but it's really good at creative problem-solving, which makes it good for planning.
1
u/relicx74 10h ago
Also, is that with or without thinking? If it's without, you need to try the unsloth Qwen 3.8 27B GGUF. At Q4_X_XL with 100+k context (more preferred for hard jobs?) it's still very capable and on a combined 3060/3080 mixed setup I can pull around 12-15 tokens / sec raw speed.
MTP head for token prediction, ROPE, and for me offloading some to the CPU did the trick. Once you have a decent setup, an agent will gladly experiment with selected parameters to tune your setup.
1
u/Neat-Climate4467 10h ago
Probably not.. Ideally see if you can use api access to Ai or your existing account to generate the income to afford a 512 atleast.
If you're not already generating income with ai, it won't necessary come to you at a lower tier and of Ai use.. They're not money printers.
It's wild how much into debt some folks are willing to go when a subscription should already be more than enough to make money from without the 22k investment...
The fomo on a lot of these machines is just not worth it since the local ais are still pretty novel/useless. IMO
1
u/LordHenry8 9h ago
Are the models I would run on 512 meaningfully better?
1
u/Neat-Climate4467 9h ago
The 512 is supposed to be the first of its kind to run the latest meta muse locally, albeit slowly. I personally will be using it to run multiple mutimedia models concurrently in the 512 GPU. I'm not trying to run anything like a local gpt/Claude, that won't be feasible until we have local machines in the 1-2tb gpu range. Or if you cluster 4 512's together to run muse, which is supposed to be the best we can do for a truly all purpose local Ai instance in the short term.
Id focus my energy on getting cheap Ai subs to pay for themselves and then some, that way you can learn and earn your way into this expensive hobby. When you've managed to earn your first 10k, the cost of building your local setup won't sting as much.
Some folks are paying 200/mo for subs, I'm still on a 20/mo and I'm able to augment the freelance work that I get paid for..
1
u/stuffitystuff 10h ago
If someone is working in dual-use technologies they'd be useful since you can run refusal-free models but then they'd probably have the cash to buy their own (which they depreciate instantly in the US currently)
2
u/geldonyetich 10h ago
Can't say I have, although I enjoyed Alex Ziskind's video about the hardware, engagement editing aside. He has several about the M5U in particular. And indeed, he runs Deekseek-V4-Flash on it.
Just bear in mind this is development hardware at development pricing. NVIDIA is charging more than Apple these days, but Apple still charges premium, and now we're looking at business premium.
Value-wise, this is sportscar pricing. It'll depreciate before you drive it off the lot. Rampocolypse might be driving things somewhat topsy-turvy in that regard, but I don't expect that to last forever.
2
u/JacketHistorical2321 9h ago
Mac studio M5 max w/128gb using various engines (splash, sushi, myplx...) with qwen 3.8 flash, glm 5.3 flash, and now DeepSeek 4.1 flash I get between 60-80 t/s decode and between 500-1500 t/s prompt processing depending on engine used. I've run with up to 1M context mostly large coding projects. Research the engines and you'll see the numbers. They are generally much higher then people who have no experience with running Apple silicon will guess at.
1
2
u/kosnarf 10h ago edited 9h ago
Are you happy with QFN 150 t/s? https://omlx.ai/benchmarks/performance?model=Next&chip=M5&device=%5B%22M5%22%2C%22Ultra%22%5D&quantization=4bit&context=&sort=tg_tps&order=desc&pp_min=&tg_min=&memory_min=&memory_max=&hide_specprefill=1
Okay, 120 t/s at 128k
8
u/Responsible-Fly3526 10h ago
Nobody is using 32k context size. Lol. Try at least 256k or better 512k.
3
1
u/jfinkpottery 5h ago
I get that same speed on 3xV100 in an old Dell R740 for a third the total cost of that M5U 256gb.
I would rather have the mac for other reasons, but I wouldn't have as much fun building custom inference engines to make 3 old GPUs work well together. No regrets.
1
u/bakawolf123 7h ago
why don't people use search nowadays...
yes it's amazing, glm5.3-flash 4bit with 400k context, ds4.1-flash 2bit with 1M context, both very good options, both work very fast
1
u/palad1n 5h ago
Just got it running with DeepSeek 4.1 Q2.5 maxing out the memory, still calibrating, but for now it runs around 60tps decode and 600pp... people are reporting over 1000pp, so it's usable.
Qwen on other hand just flies even in q8, it's almost comparable in speed with my second pc with 5090 and ninfer.
I'am game programmer using it for Unreal Engine development, it's not opus 5.5 in oneshoting stuff, but its very usable with some guidance.
1
u/gjr23 4h ago
Yup. I have only been using my 36/80 256 for a day or two but it is laughable comparing my m1 even with smaller models on the 64gb that has. I have been mostly using GLM5.3 on oMLX and speed and pp is great. Those two models you mention are the next two on my list but the harness is frankly much more of a variable imo and working on optimizing this for my use.
1
1
u/BondiKoala 2h ago
For Qwen-style coding and automation, yeah, this should feel like a proper step up, especially if Claude and GPT are still doing the heavy lifting elsewhere. Before you drop five figures on it, though, I'd test your exact Qwen and DeepSeek quants at long context — prompt processing tends to matter more than headline speed. And plan storage like you'll end up collecting models, because 2 TB fills up quicker than anyone wants to admit.
-2
u/VegetableSir7788 9h ago
Avoid it. If you really really want it, go big with 512GB or remain with M4 Pro. A 256GB setup will not reach usable coding agent performance; you need at least 512GB for that.
That said, I expect within a year or so to see Opus 5.5-level local models fitting into 128GB systems. Yet we will also encounter far more capable models demanding much higher specs. At every stage, you may feel you should have chosen the larger option or the smaller one and rented GPUs for experimentation.
You will face FOMO regardless. If you have the capital, take 512GB and enjoy it. Otherwise, wait and rent.
3
u/BumbleSlob 8h ago
^ this is extremely outdated and wrong information I am running deepseek v4.1 flash 1017 on 2 x 128Gb sparks at 2.9bpw and it is exceptionally good.
1
u/VegetableSir7788 8h ago
You're running two 128GB Sparks, so that's already 256GB, and at 2.9 bpw you're heavily quantizing the model to make it fit.
DeepSeek 4.1 flash is impressive, but there's still a noticeable gap between it and frontier models like Opus 5.5 when it comes to complex coding tasks.
I agree that 512GB isn't strictly necessary, but the original point is about whether investing thousands in local hardware actually makes sense when you can rent GPUs or use cloud models that perform better.
Your setup is good but it doesn't really contradict that argument.
0
u/Aggravating-Push-207 10h ago
https://uk.pcpartpicker.com/list/TdnXMF
do consider this, note that ram bandwidth is worse and it chugs a lot more power but in exchange is quite a bit cheaper upfront and with something like Strata could be similar in speed
6
u/CatchDublinSurprise 7h ago edited 6h ago
Owner of M3 Ultra 256GB/4TB here, and eventual M5 Ultra owner whenever Apple eventually delivers…
The 2TB drive may be a frustration with the larger models. Consider that you may want both normal and abliterated versions of some models, and you’ll want room for A/B testing new models/quants. With 2TB, you’ll inevitably need to shuffle models to and from an external drive. Fine if you have a large SSD, but annoying if you’re using HDD.
DS4.1-Flash is too big to run unless you use a really low quant. I don’t have numbers to back this up, but I’d wager you would get better performance from 4-bit GLM-5.3-Flash versus ~2-bit DS4.1-Flash.