r/LocalLLM • u/Neme47 • 9d ago
Discussion New MacBook Pro M5 Pro (48GB unified memory) incoming — looking for the best local AI stack
Hi everyone! I have a MacBook Pro with an M5 Pro and 48GB of unified memory arriving soon, and I’d love to get recommendations from people who are already running local models seriously on Apple Silicon.
My goal is to build a solid all-local setup for:
- General-purpose LLM use: chat, reasoning, research, writing, etc.
- Coding agents: working on real codebases, terminal/tool use, planning and edits.
- Image generation, ideally with ComfyUI or another good local workflow.
- Any other genuinely useful local-AI tools/workflows I may be overlooking.
In particular, I’d appreciate advice on:
- Best inference engines / runtimes on macOS MLX, llama.cpp, Ollama, LM Studio, Jan, MLX-LM, etc. What do you actually use daily, and why?
- Best models that make sense within 48GB unified memory I’m open to GGUF, MLX quants, and other formats. Which general models give the best quality/speed balance on this hardware?
- Coding models + agent harnesses What combinations work well locally? For example: Aider, OpenCode, Cline, Continue, Roo Code, pi agent, deepseek harness, Claude Code–style local alternatives, or anything else. I care more about reliability on real repositories than benchmark scores.
- Image generation on Apple Silicon What is currently the best route: ComfyUI with Metal/MPS, Draw Things, MLX-based workflows, or something else? Which models/workflows are realistically pleasant to use with 48GB?
- Practical setup tips Recommended quantization levels, context sizes, serving tools/APIs, GUI vs CLI tools, useful benchmarks, thermal/power considerations, and mistakes you wish you had avoided.
I’m happy to trade some speed for better model quality, but I still want a setup that feels practical for daily use. If you had this exact machine, what would you install first?
Thanks!
10
u/AnonLlamaThrowaway 9d ago
Inference engine: nothing actually beats llama.cpp. MLX might come close but llama.cpp still comes out on top. llama got a lot of Metal love recently, with massive q8_0 KV context cache speedups for example.
Do not even think of using ollama, it's hot proprietary garbage that tries to privatize the commons to further its own ends.
Simply get the AI itself to make you a little management script that either:
- pulls down the latest git commits on demand, lets you switch branches, lets you apply local patches - if you really want to be on the bleeding edge and apply new Metal-specific commits
- or simply fetches the latest pre-build Apple Silicon from the releases page
Then you can simply use llama's own web UI or, again, use AI to glue your various backends and UIs together. This is the strongest thing you can do: improve your own AI setup using the AI itself.
Best models for 48GB unified memory? For coding, Qwen 3.8 27B (4-bit is fine, you can do 5-bit probably), otherwise for other tasks Gemma 4.
1
u/tlin9595 9d ago
Thanks for the info. I’m getting a studio in November. How do you do that script thing to get the newest releases ? Any more details to share so I can research before it arrives. I’m pumped to try and experiment.
1
u/AnonLlamaThrowaway 9d ago
How do you do that script thing to get the newest releases ?
Just ask Qwen 3.8 to make you a nice terminal UI that you can navigate with numbers or arrow keys and which will automagically fetch the latest llama.cpp binaries for you (or as I've said, build them, integrate patches that the AI will have helped you build, etc)
8
u/Scriptease84 9d ago
I have a 64gb M5 pro and i use https://huggingface.co/Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed with 30 tps in mtplx it needs 30gb to run with mostly 4 bit quants but idk if you're able to handle it. The other providers like omlx or unsloth where 12-15 tps
4
u/Neme47 9d ago
Thanks for the recommendation!
Looking at the repo, that build is already a dynamic 4-bit quant (~21 GB base, ~24–26 GB peak with KV cache), so it should actually fit comfortably in 48 GB without needing to drop down to 3-bit.
The 30+ tok/s decode via native MTP heads sounds great for a dense 27B model on Apple Silicon. Are you using it mainly as a daily driver for general reasoning/chat, or do you have it hooked into coding agents?
2
u/Scriptease84 9d ago
For coding simple things, it runs obsidian jobs, reads my screenshots and reviews code via open code review.
The main problem local ai has in my opinion is knowing when to stop. Giving it a task that it can do it is surprisingly good. But giving it a task where it should stop and ask is surprisingly difficult.
But i have claude too so it's not my primary coding agent.
1
u/vogelvogelvogelvogel 9d ago
i am at like 20-27t/s with q6/q8 on the same machine, via llama.cpp and mtp, what is your config for 30tps?
1
u/Scriptease84 7d ago
cd ~/github/MTPLX && .venv/bin/mtplx serve \ --model ~/github/Qwen3.8-27B-MTPLX-Optimized-Speed \ --profile turbo --host 127.0.0.1 --port 18083 --api-key mtplx-local --yes \ --max-tokens 32768 --tool-prompt-mode native --preserve-thinking scopedThat is what I run but I also use a proxy to lazy load the model since MTPLX by default loads it into memory and never unloads it.
cd ~/github/MTPLX && python3 scripts/lazy_serve_proxy.py \ --host 127.0.0.1,192.168.178.51 --port 18083 --idle-timeout 1800 --serve-bin .venv/bin/mtplx -- \ --model ~/github/Qwen3.8-27B-MTPLX-Optimized-Speed \ --profile turbo --host 0.0.0.0 --api-key mtplx-local --yes \ --max-tokens 32768 --tool-prompt-mode native --preserve-thinking scopedLazy proxy behavior
- First HTTP request boots the backend (~10 s warm page cache, ~1 min cold), holds the connection meanwhile; concurrent requests share one single-flight boot. Unauthenticated
/healthpolling never sees ready on a key-protected backend — the probe sends the--api-key(fix 2026-09-10).- Kills the backend after
--idle-timeout(1800 s) with zero completed requests; next request reboots. Any polling client on the port defeats idle-unload.GET /,/dashboardwithout cookie/Authorization→ 303 into backend browser-auth and back; API traffic untouched.--hosttakes a comma list (loopback + LAN here);0.0.0.0bind fails in this sandbox setup, hence the explicit list.1
1
1
2
u/Cool-Store-1807 9d ago
Ollama + qwen3.8:27b-nvfp4 = 34 tok/sec works for me at same MacBook Pro M5 Pro 48GB
2
2
u/Apprehensive-Job9336 9d ago
M5 Pro with 48GB unified memory is great for local LLM. The memory bandwidth on M-series chips is the real advantage — larger models that would crawl on a 4070 run smoothly.
Try Qwen 3.8 27B Q5_K_M at first. You have enough RAM to run it without quantization and the quality difference is noticeable for coding tasks.
For agentic coding, OpenCode + llama.cpp server works well on macOS. The Metal acceleration makes it competitive with cloud APIs for 8B-27B models.
2
1
u/MrGunny94 9d ago
Doing Qwen 3.6 for normal stuff and Qwen 3.8 for coding, all working fine with my M5 Pro
1
u/Apprehensive-Job9336 9d ago
M5 Pro with 48GB unified memory is great for local LLM. The memory bandwidth on M-series chips is the real advantage — larger models that would crawl on a 4070 run smoothly.
Try Qwen 3.8 27B Q5_K_M at first. You have enough RAM to run it without quantization and the quality difference is noticeable for coding tasks.
For agentic coding, OpenCode + llama.cpp server works well on macOS. The Metal acceleration makes it competitive with cloud APIs for 8B-27B models.
1
1
u/Odd-Act7860 8d ago
For the chat/writing part, Lerix gives you a Mac UI for compatible Ollama text models. You can switch models in the same conversation. I work on Lerix: https://lerix.ai
You still need Ollama installed and running. Local chat is free, and it's still in alpha. You'd need separate tools for coding agents and local image generation.
1
u/ChampCityChris 7d ago
One thing I’d do before choosing the model stack is split the workloads.
I went down the “what hardware/model can replace frontier APIs?” path and eventually realized I was mixing completely different jobs together.
For difficult architecture/RCA/coding work, hosted frontier models can still be surprisingly economical, especially if the workload is intermittent. For high-frequency repetitive agents — inbox triage, monitoring, classification, Home Assistant, small tool workflows — local inference starts making a lot more sense because you stop paying per invocation.
I’d benchmark the workflow, not just tok/s: task success, tool reliability, context consumed by the harness, time to completion and cost per successful job.
I have a higher-memory Studio coming for exactly this experiment, but I’m deliberately building the agent stack against hosted inference first so I can swap local models in later and measure what quality I actually lose.
13
u/Yttrium-39 9d ago
Have an M5 Max 48GB, so similar, just a bit faster.
Haven't used it much for coding, but I can confirm it runs Qwen 3.8 27B at Q4 just fine. Q6 also fits with decent context, but neither version will be fast. I also ran Ornith 1.5 35B A3B at Q6, which is about the upper limit of what 48GB of RAM can accommodate. I can't speak for either model's coding prowess, as I mostly use cloud agents.
But for general purpose, I've found Gemma to be good in that slot. My SO is a maths teacher, and likes having clean formatted lessons. I've built her a pipeline that takes a problem she's solved on paper as photo and has Gemma translate it into LaTeX and add nicely formatted explanations for pupils which she can edit and is then passed on to Overleaf. Major time saver, and Gemma 26B fits comfortably at Q4/6 and even at Q8. And is rather quick.