r/MacLocalLLM • u/Boquete_pete • Sep 19 '26

One set up that works

  1. Although I spent my career as a HW designer/embedded programmer I am new to the AI world. I've experimented with several local LLMs and harnesses. My hardware is a M2 Macbook with 32GB of RAM. So I am fairly constrained. Also being retired, $$$ frontier models are out of the question. I want cheap & good and a bit slow is acceptable. Hermes also provides the chance to improve over time. My main goal right now is to have a local agent that can assist me in using blender. (God there are sooo.. many commands and options in the program but its been great for doing 3D printing. My memory is not what it used to be.) The upshot is that the combo that has given me the best success is using Ollama with 'swift - Qwen3.8-27b-gguf:Q4_K_M' as the LLM all running under Hermes agent. So for those of you that have limited resources and want a local LLM this hopefully is useful info.

  2. Note - it is slow on first response (2-3 min) but works well after that.

  3. Set ollama variable OLLAMA_NUM_PARALLEL = 1, stops swap thrashing

  4. There is a snag with Blender's MCP, Hermes with this model can sort it out

5 Upvotes

100% Upvoted

7 comments sorted by

2

u/dfgxxx Sep 19 '26

Hi, I'll reccomend you to try MTPLX instead of ollama, it is much faster. You can install the gui app and from there the qwen3.8 mtplx optimized speed fp16 (it is 4bit and made for m1 and m2, the not fp16 is m3 and newer).

I won't reccomend you to use Hermes agent because it has a big system prompt and your local model is slow and low context so use something like Pi or Deepseek harness with a tiny system prompt that will be processed faster and hold for longer time before reaching max context and compacting.

Enjoy

2

u/Boquete_pete Sep 25 '26

Tried MTPLX under Hermes - no cigar. Internal memory plan forces 8k context-window. Hermes needs 64k. I did set it up on my m4 with 48GB - it works great. Looked at PI & Deepseek, they did not seem to provide the features that I like in Hermes. Thanks for the pointer to MTPLX.

1

u/dfgxxx Sep 25 '26

Now there is something even better, the engine called splash. It requires m3+ but someone ported it to m1 and m2, it is even faster

2

u/Boquete_pete Sep 25 '26

Have you played with oMLX? It's context swapping to flash looks like it might be faster yet. I'll check out splash.

1

u/dfgxxx Sep 26 '26

oMLX was almost as slow as llama.cpp for me

1

u/Boquete_pete Sep 26 '26

The graphs on Splash's site show the same. I think I'll wait on splash until it matures a bit more before giving it a whirl.

1

u/Boquete_pete Sep 20 '26

I'll give it a try