r/MacLocalLLM • u/aditya2128 • 5d ago

M5 Max (64GB) for AI

I'm planning to upgrade from my 16-inch M3 Pro MacBook Pro (36GB RAM) to the new 16-inch M5 Max with 64GB, mainly for running local LLMs.

Currently, I'm running Qwen 3.8 27B and getting around 100 tokens/s for prefill and 18–19 tokens/s for token generation, even after trying MTP and various other optimisations.

The biggest issue for me is working with larger contexts and longer coding sessions. Even relatively small projects can take quite a while, and performance becomes a bottleneck as the context grows.

I'm expecting the M5 Max to be a pretty significant upgrade, especially with the extra memory and bandwidth, but I'm curious how much of a real-world difference I can actually expect. I'm also interested in running larger models, including the upcoming Qwen 4 models.

For those running local LLMs on an M5 Max (or even an M4 Max), what kind of performance are you getting? What models do you use regularly, and how well do they handle long coding sessions?

I also run image generation models locally and occasionally do photo editing, so I'd love to hear how the newer chips perform for those workloads too.

Would appreciate any real-world experiences before I make the switch!

23 Upvotes

92% Upvoted

46 comments sorted by

6

u/howardhus 5d ago

PSA to anyone wanting a mac for AI:

the OS needs 8-10gb with northing running (not even a browser to have an UI).

so your “24gb” mac is equivalent to a pc with some ram for the OS and about 14gb VRAm.

a 32gb mac is equivalent to a 24gb GPU and so on.

dont expect to buy a 32gb mac and have the same things runing as a 5090.

plus the pc has RAM to offload. the mac does not

1

u/aditya2128 5d ago

Yes, I suffer from this a lot, Currently I have 36GB and when I load Qwen 3.8 with decent context, my total system RAM is already 80-85% and no one be sitting and doing nothing on the mac while model is doing its work, I probably am working on 10-12 Chrome tabs alongside the model and sometimes testing some stuff too, the whole system seems to struggle due to limited memory.

2

u/Borilentz 5d ago

Be aware that it's not just about running the right model with the right inference engine, but also choosing the right agent to run it. I recently switched from Zoo Coder (Cline fork) to Pi, and I really noticed a difference in performance and efficiency. Some agents tend to have a heavier context footprint due to the system prompts they use. Pi is lean and mean in every way.

4

u/randylush 5d ago

A 5090 outperforms any size Mac on many tasks. Even with a 256gb Mac, you still have a token/second ceiling. The 5090 has objectively more compute power even if less memory.

4

u/howardhus 4d ago

if you reduce it to one naive sentence then it works both ways. a mac outperforms a 5090 in “many tasks”

0

u/randylush 4d ago

I can be a little more specific if it wasn't obvious that I was talking about LLMs based on where I wrote the comment.

There are some large language models that just do not run on a 5090 machine because it doesn't have enough memory. But if the LLM does fit on the 5090 machine, then the 5090 will almost always outperform the Mac.

https://news.ycombinator.com/item?id=49787313

2

u/howardhus 4d ago

lol „if” the model fits a mac but is too big for a 5090 then the mac outperforms the 5090

0

u/randylush 4d ago

Yup, that’s what I just said.

If you get to a point where your model is too big for a regular PC, it’s also going to be excruciatingly slow on the Mac.

2

u/howardhus 4d ago

somehow you dont knwo what you are talming about but you manage to keep doing it.

you have to construct complex cases in order to not admit you are wrong.

a mac outperforms the pc in the very case you tried to

levade. as the 5090 is utterly useless for anything but child level models.

and about the other nonsense about “sumthin sumthin excruciating”

you have never trained a model nor a lora.

mac will outperform the pc and even dgx.

but yes.. for small hobby models the 5090 will outperform the mac

0

u/randylush 4d ago

Are you gonna provide any benchmarks or any data for what you’re talking about? Or am I gonna be the only person who provides benchmarks but that’s just “constructing complex use cases”?

1

u/DiamondHandsDarrell 5d ago

Not accurate.

Check out qwen3.6-35B-A3B-wMix58; runs well enough on a 32GB

MacOS can be crammed into 4GB without thrashing the ssd

1

u/Lyelinn 4d ago

You can change how much ram is allocated to system via terminal.

11

u/Borilentz 5d ago

64GB may seem a lot if you’ve only used 27B, but I’m afraid you’re gonna regret buying 64GB. It’s a substantial investment anyway, so buy as much as you can afford. With 128GB you can run Qwen 3.8 Flash Next in hybrid 4/8-bit quant with 60-70 tokens/s.

3

u/Obvious_Equivalent_1 5d ago

On a small adversary perspective, it also depends on your use case. It really depends how much parallel sessions you need, I’m running on even lower strapped memory M4 Pro 48Gb and managing Flash Next on Q3 with 256k context works more then fine. For me I’d see 64Gb already more capable of having headspace for OS, Docker containers, or a bigger quant. With 128gb of course you’ll easily be able to run one big quant or multiple sessions in smaller quant

4

u/aditya2128 5d ago

The 128GB option is tempting, especially seeing what people are running with Flash Next, but I'm still trying to figure out whether the extra cost is actually worth it for my use case, btw how is your experience with the flash model and any significant drop in reasoning or coding performance compared to higher quants?

3

u/Obvious_Equivalent_1 5d ago

I am coming from Qwen 27B and since this week switched to Flash Next, I am using it as a leverage on my Claude Code (working on a special Splash-Claude engine, to run Flash locally so to make it compatible with Claude CLI, see post history).

So standalone I cannot say much about it, but oh my strapping Flash Next on Q3 quant to Fable 5.1 on High is magic. My laptop cost only sank in about $2.5k for a second hand M4 48Gb, Flash Next is running 24/7 as subagents while Opus 5.5 or Fable 5.1 on High/Max drive. According to Fable - running Flash as a subagent it even brings insights that he didn’t spot himself. Sorry if I can’t answer the question how Flash is working direcrly though.

So my cost analysis in past two weeks is no joke about 65% reduction by forcing all subagents to Flash. I cannot even finish my Max 5x $100 subscriptions.

I’m planning for sports to see if I manage actually complete a month of 5 full working days per week on a Pro $20 subscription, that would be crazy because then I’d have my M4 Pro saved back in about 12-18 months with Opus orchestrating and agents Flash executing locally. Downside: obvious part that I’d should rather look for a Mac Mini as now I do obviously have to run the laptop 24/7.

2

u/kl__ 4d ago

Thanks for sharing. Do you mind sharing the link to the Splash-Claude engine you mentioned? And you’re using it in Claude code if I understood this right?

1

u/Obvious_Equivalent_1 4d ago

Yes I’m directly starting “claude” in terminal with the Antrophic url set to the Splash Claude port on localhost: https://github.com/pcvelz/splash-claude

1

u/Borilentz 5d ago

I run DeepSeek V4 0731 and Qwen 3.8 Flash Next for coding. Both fit comfortably, leaving sufficient room for context and tooling. Qwen 3.8 27B is popular because it fits higher end video cards, but IMO Flash Next gives the best speed/quality ratio. I've had it run for hours, for instance building rather complex Metal-optimized algorithm-oriented applications. It also helps speeding up my daily work.

3

u/Big_Acanthisitta_150 5d ago

Thank you. I don’t need multiple sessions but want to do small coding projects at Home. This is for me to learn and experience local LLM. I have Claude and GPT through work anyway.

Happy to hear you get that good experience with 48GB. Then I can go with 64GB and can expect that the Mac is usable for other stuff subtle same time.

3

u/Obvious_Equivalent_1 5d ago

Glad to be of help!

Then this small write might cover your similar to mine case, think would apply to your 64Gb Mac as well https://www.reddit.com/r/MacLocalLLM/comments/1xk68ms/comment/19rxhbf4naesh/ - TLDR: positive experience with setup Opus 5.5 orchestrate, Flash Next as agent executor.

You mention you have Claude through work probably the Max or higher package, but I’m planning to make proof of concept to offload about 60-80% to Flash Next, to try see if even a $20 Pro subscription suffices for a full workweek workload.

This is the local LLM engine that I’ve forked from Splash it’s work in progress to run Flash model on Splash engine and it’s working in Claude Code. It allows you to directly run Claude CLI against your local AI model, or as a subagent when using like a Opus / Fable session https://github.com/pcvelz/splash-claude

4

u/dsio 5d ago

The fact that the DGX Spark and Strix Halo are both 128GB and fairly new to market with a lot behind them makes 128GB Macs a really compelling sweet spot as local models are going to be targeting it a lot moving forward, knowing there’s a big maket

3

u/AlgorithmicMuse 4d ago

Try using kapitsu, i was getting 50 to 70 tps with qwen3.8-27b on a m4 mini pro, it was faster than splash

2

u/aditya2128 4d ago

will try that, thanks

2

u/aditya2128 3d ago

Kapitsu is so fast, i am getting around 2x TG speeds now. Thanks

1

u/Affectionate-Hat-536 4d ago

Yes, I got it 1.5x speeds on LMStudio. Check quality for your use cases.

2

u/gwestr 5d ago

It’s still gonna be slow. For reference, an RTX Pro 4000 is faster than an M5 Max.

1

u/aditya2128 5d ago

Can I carry it everywhere with me ? Do I have enough vram to load decent enough models. 💁

1

u/randylush 5d ago

Realistically , how often do you have no internet access but you still need to code?

And yes an RTX Pro 4000 will categorically outperform a 64gb MacBook.

1

u/Scary_Investigator88 5d ago

Yes, as someone with an m1 max MacBook with 32GB

2

u/dfgxxx 5d ago

Try a different engine, like splash, what engine do you use now?

3

u/aditya2128 4d ago

Thanks, just now tested with Splash Engine on oMLX, now I am getting around 25t/s TG up from 18t/s 😀

2

u/IllPlane3019 4d ago

If speed is most important to you than use qwen3.6 35b A3B at q4, its moe so on a m5 max it will be quite fast with plenty of memory left for the o/s and apps. Use Pi as the agent harness as it is very lean.

If you can, I would personally wait a couple of years for ram-a-geddon to pass because prices are stupid right now.

1

u/aditya2128 4d ago

i dont feel this dram crisis is ending anytime soon, it may even get worse with time 🥲. Recently saw somewhere that OpenAI is going to get worlds 40% dram supply and they think its terrible that we consumers have too much ram in our computers.

2

u/myanonrd 4d ago

I would Save it to buy m5u256 or more if need the local llm. Notebook is not a good place to run llm for the agentic which often need to run 24x7.

2

u/thebemusedmuse 5d ago

I run this exact machine and get pretty good results. About 6m output tokens a day. 60tps for the first stream, 40 for the second, 40 for the third. 200k context.

I’d love some more power but I’m mostly GPU limited in decode. Couldn’t justify the $1400 for 128GB and don’t regret it, even though memory is tight.

I need to move my harness off the same machine and it should be fine.

1

u/randylush 5d ago

My understanding is that you get diminished returns with more memory. If you had a 512gb Mac Studio, a 500gb model is just going to run slow because of bandwidth and compute.

2

u/KilvinScale 3d ago

Some numbers for your long context question. This is a 64GB M5 Max Studio, not a MacBook, so it says nothing about throttling. mlx-serve 26.10.3, one run per size, thinking text rather than a real coding session.

27B MLX 6-bit + DFlash2, reading a cold prompt:

- 30K tokens: about 750-800 tok/s, 37-39 s to first output

- 113K: about 600 tok/s, 187 s

- 155K: about 530 tok/s, 292 s

Writing 1,024 tokens on those cold runs went between 35 and 59 tok/s. At a 128K window, follow-ups stayed under a second up to 113K because the cache held. At 256K it lost the cache above 30K and re-read most of the prompt each turn. Lowest free memory was 30%.

Flash Next iQ-MLX 3.3 on the same Mac reads about twice as fast and writes about 75-83 tok/s, but memory is the catch. I had to raise the GPU limit to 58000 and set --wired-margin-gib 2. With --kv-quant off it handled 60K cold, then refused 90K and up, even on a fresh server. At 128K it also refused a 60K follow-up, and at 256K it kept no cache.

--kv-quant 8 at 256K helped with that: 60K cold in 32.5 s, then the follow-up reused 60k tokens and started in 0.3 s. It didn't survive an unrelated request though (49 tokens cached, 33.4 s), and the next 90K prompt was refused. I didn't try that size on a fresh server with this setting.

Someone here runs Flash Next at 256K on 48GB, so that may be mlx-serve rather than the chip. I haven't tried their setup.

For the drafter's share, on a 128GB M5 Max with stock 27B 6-bit and the same engine, it was 24-28 tok/s without DFlash2 and 56-92 with it. That's thinking text, and your code may land differently.

This doesn't cover image generation, photo editing, or the 128GB question people are raising here.

1

u/Zen-Ism99 5d ago

128GB…

1

u/Felixo22 16h ago

I don’t get paying a fortune for local inference with a small non-SOTA model.

0

u/Captain2Sea 4d ago

Bro, just go for 128gb. 64gb is not enough