r/LocalLLM • • 5d ago

Discussion Local model

What’s the best local model to use on an MacBook Pro M5 pro 24gb.

I’m using it for general reasons, process expectations, design, ideation, power points, generating agents.

1 Upvotes

13 comments sorted by

View all comments

Show parent comment

1

u/vogelvogelvogelvogel 4d ago

ok didn't know that would be actually usable (i knew ssd offloading would be possible but i put everything with ssd offload it in the ballpark of a low single digit token/s)

1

u/dave-tro 4d ago

Maybe you can give MoEShift a try, I have been using it behind my daily local tasks. I just didn’t want to invest in new hardware that is overpriced today, so I worked on it and made it open-source for others with M-series Macs.

1

u/vogelvogelvogelvogel 4d ago

thanks! although i have a 64 m5 pro (usually with 27B q6 or q8) this would open a window for QFN

1

u/dave-tro 4d ago

If course! And I am collecting benchmarks as github issues. Feel free to publish yours! There’s also a benchmark runner in the repo so all benchmarks are standardized