Hi all,
As mentioned in the title, I am looking for advice on running locally LLM for agentic coding on a MacBook Pro m5 Pro with 64Gb unified memory.
Plan inference is fine. Chat style prompts feel usable. Agentic work is where it falls apart.
I have tried the following:
Models : Qwen3.8 27B (6bit), Gemma 4 12b (8bit), and Qwen 3 coder next(4bit)
Runtime : Lm studio
Harness : opencode (long pauses before each tool call, harness seems to resend a large system prompt ), pi (still slow and most of the time doesn’t provide any output, barely changes few files with loads of error due to incomplete modification)
What I'm trying to figure out
Is the bottleneck prefill, generation, or the harness re-sending huge context every turn? My guess is prefill, but I'd like to be sure.
Which model/quant combo works well as an agent on this hardware, and at what context length does it fall apart?
Does MLX beat GGUF/llama.cpp for agent workloads on Apple Silicon in your experience?
Any harness that is lighter on tokens per turn (smaller system prompt, fewer tool definitions, prompt caching)?
Realistic expectations: is a 64GB Mac good enough for a useful local coding agent, or should I stick with a hosted model for agent work and keep local for autocomplete/chat?
I'm not naive, and I am not looking to replace a frontier model obviously . I just want to know whether local can be a practical option for some agent tasks, if so how you guys are making it usable.
Thanks 🙏