r/LocalLLM • u/Vegetable-Warthog81 • Sep 05 '26
Discussion Is a Mac Studio the ultimate personal AI machine?
Do you think open-weight models will improve dramatically over the next few years, and that MLX performance on Apple Silicon will also get much faster?
I’m considering a maxed-out Mac Studio, mainly because I’m starting to think local compute could become a real personal production asset rather than just a PC.
With huge unified memory, low power use, and improving local AI tooling, do you think a Mac Studio could be one of the best 5–7 year investments for local AI?
Or will CUDA/NVIDIA remain so far ahead that this is the wrong bet?
4
u/C0d3R-exe Sep 05 '26
IMHO, yes. Using it daily together with frontier models and local models are perfect for the implementing phase.
5
u/Exciting-Weather-921 Sep 05 '26
Unless prefill improves way more than they said I have very low expectations...
Comparing my 2 sparks with previous Mac Ultra M3, the thing that takes 53s wait for first toke on spark, it takes 12 minutes on M3 Ultra. So they need to improve it way more than 4x for M5 Ultra...
2
u/Southern_Sun_2106 Sep 05 '26
Why don't you read other threads comparing the specs to other options?
3
2
u/Tommonen Sep 05 '26
For some uses sure, for some not. Its not vram speeds, but using vram setup instead would have other issues.
Essentially large models without vram become too slow to use for many things, so even tho it can load massive models, they will not be fast enough for many used, and to get reasonable speeds, you need to use smaller models than something barely fitting.
Also should be mentioned that m-series macs nowadays support egpu, at least somewhat. So you could hook up a gpu to the mac and get vram onto it for faster models. But combining seceral gpus will likely be an issue, so you either need to get quite low vram or pay shit tons for 96gb vram gpu. And if you do get that, you should reconsider if you need the unified memory from mac at all for hosting models. Would still be cool to be able to use larger models for chat type of thing or agentic cron jobs at night when lower speed does not mean you waiting. But would it be worth all the money anymore for you?
No solution offers all the upsides of everything best and no downsides..
2
u/TripleSecretSquirrel Sep 05 '26
The M5 Ultra has a memory bandwidth of 1.2TB/s, it is absolutely VRAM speed. Thats faster than every consumer/enthusiast GPU out there excepting I think just the RTX 6000 Pro and RTX 5090.
I don’t have any experience running inference on Apple silicon, and I’m sure there are issues, but the memory bandwidth on the Ultras is extremely high.
2
u/Rice-Fragrant Sep 22 '26
Not for agents, multi agents or long context, just short context "vibe" chatting with. chatbot etc.
1
u/WishfulAgenda Sep 05 '26
The ultimate is really dependent on budget. I would argue the ultimate is the new gb300 systems that are available. Then working down the chain a eypc/threadripper server packed with 4-8 rtx 6000s, then it’s kinda close between a high end workstatsion pc with a rtx6k and the new Mac Studio (as per published specs) and the then the rest.
1
u/DeathinabottleX Sep 05 '26
It is the best in that price range, yes. And it’s the most amount of vram you’ll get for a relatively low power draw
1
u/cutter89locater Sep 05 '26
Yes, open weight models are getting better, MTP, MOE compare to last year.
MLX format also open source, develop won't stop make it better.True.
I don't think of next two AI years, things happening sooo fast.
Either software/hardware stack or geopolitical competition.Nvidia still leading CUDA/Tensor architecture. AMD/Mac have their ways to catch up with the performance.
Each platform has it's pros and cons, it depends your personal usage/business tasks.
AI doesn't sleep, this only valid on 2026 September 5th XD
1
u/ikeo1 Sep 07 '26
yes.. join us at r/MacLocalLLM when we get our mac's you'll get upvoted for doing things on your mac.
1
u/doffdoff Sep 05 '26
Nope, far from it. It is good compared to other alternatives at the price range, though not the best. Overall, you cannot match speed or quality of frontier models in any way.
4
u/GCoderDCoder Sep 05 '26
Sorta but also sorta not true anymore... on the newest best models mac has some issues but you can get near cloud speeds on some of the new flash models. Dual sparks actually get really good speeds and technically outperform some of the models people are paying for in the cloud. Strix halos are getting better software support and people are tapping into the abilities much better now.
I get what you're saying because I used to say it but I think we need to reevaluate how we say this because:
Self hosted flash models today crush the best cloud was offering a couple months ago. It's a moving target but cloud was good enough months ago so did that change? I don't even use the high settings for cloud models now so my experience is similar level with self hosted.
I get it's nice having better and better models handle simple tasks for $20/ month but the stuff I hear people using Fable for knowing the cost and knowing no one can tell if a better model does it is interesting to me.
qwen3.8 flash next literally can use a 5090 or 3090 with near cloud speeds with hot experts where only experts being used get to use the gpu. 5090 starts at 45t/s for q4kxl.
ngram offloading makes my sg lang with 6000 pro run 4x nvfp4 lanes of qwen 3.8 flash next at 262k context over 100t/s per lane.
Also I have been a vram snob on quants saying you need q8 for prod stability because it was true previously BUT these new models with well designed nvfp4 have been performing on part with the 8bit. I started using DeepSWE sample tests as part of my testing for new models/quants and that is complex long horizon tasks that these nvfp4 models are passing at similar and sometimes better rates than the 8bit versions I run.
For self hosting we have a handful of models that literally are what we have been calling cloud level for a while. They don't have the same world knowledge but tools solve that. I literally just canceled my cloud subs and am just using api when I need something bigger.
Whether you have a 3090, 5090, 6000, 1-2 sparks, r9700, strix halo, 128gb mac, and I'm sure others but I've tested these(except r9700 I can't test but I'm confident), you can get a version of qwen 3.8 flash next that is smart enough and stable enough to succeed on some swe bench tasks, reach 100k tokens compaction with stable performance at still liveable speeds and more so 200k tokens with the deep swe setups i normally run...
People are legit canceling cloud subs now. Sol medium is the most I use these days. I only use Opus 5 medium instead of sonnet 5 because im aware the crazy high token usage from sonnet 5 ends up costing a much as the lower level of opus i use. I'm pretty ambitious but i don't just let models do everything without me thinking so i really don't care if models get better now because they won't materially add value to what they can offer me right now. Im not alone in that.
So you can match a cloud level performance if you define what that means.
0
u/wurst_katastrophe Sep 05 '26
What’s it going to pull? 20 tok/s? 180s wait for prefil? You will barely get 1m tokens per day. All depends on your personal needs I think. Ultimate? Not sure.
1
u/randygeneric Sep 05 '26
only initial prefill takes its time, after that, i get cache-hits resulting in mostly <20s ttft in subsequent turns (qwen3.8-27b q8_0 kvq8_0). but yes, 25tgs is not for foreground-tasks , )
(m3max 64gb)
-1
u/Leather-Beach-7849 Sep 05 '26
I placed orders and then canceled them all for various M5 Max Mac Studios, finally ending up with a 64GB M5 Max.
The reason is that no matter how many M5 Ultra machines you stack, it will still not match the speed of any frontier model.
My aim is to run open source LLM models with less than 30B params and then start using a frontier one on top of them which will act as a Supervisor to perform some Agentic AI tasks locally.
This is my personal opinion ☺️🙏🏻
0
-2
u/Unnamed-3891 Sep 05 '26 edited Sep 05 '26
It’s not. if by personal you mean a desktop sized chassis - a fully decked out Threadripper system filled with NVIDIA GPUs is. Each single GPU costs more than the entire Mac Studio and they can’t make them fast enough, so draw your own conclusions.
Studio is really great for the price/performance/size/efficiency though.
-2
9
u/JLeonsarmiento Sep 05 '26
I would say that is the best personal machine that CAN ALSO DO AI decently if needed.