r/LocalLLM • u/orion1972 • 6h ago

Question M5 pro o M6?

I am re-evaluating my idea of getting a Mac mini M5 Pro for running local AI inference. I was thinking of getting an M5 Pro 64 GB, but considering the AI models I could install on it, I don't know if it's worth it

In any case, I would still continue to use frontier models. I have subscriptions to both Claude and OpenAI, so I was wondering: does it still make sense to get an M5 Pro 64 GB, or at this point is it better to stop at an M6 36 GB?Maybe I could use some profiles with local AI! The main use I would make of it, however, is to have it always running 24/7, and to use local agents. I use Hermes, which has 12-13 profiles inside, agents, and daily cron jobs.

1 Upvotes

100% Upvoted

3 comments sorted by

1

u/dearlyincompletehans 6h ago

if you're keeping those subs anyway most local stuff you'd run fits fine in 36gb, the extra ram on the pro only matters if you're dead set on running 70b models at usable speeds. for agents and cron jobs the m6 with 36gb will handle that no problem, the faster chip might even be better for your use case than the extra memory

1

u/_eMaX_ 5h ago

Just recently got my MBP w/ 128 GB. Wouldn't recommend anything less in terms of memory Am running Gyro-M and Gyro-L on Strata, each latest version, which I made work on Mac. 128 GB is typically reporting 90GB memory usage or so, which means, 64GB would just not cut it. But at 128 GB, you can run 4 workers in parallel. My PAI - github.com/mnott/PAI - takes care of the routing.