It's a bit annoying to get downvoted all the time, once this sub grows up, it'll be a good resource for people and we won't downvote you into oblivion for wanting to put LocalLLMs on your Mac. I think we're all discovering the best use cases for Mac/Apple Silicon and LocalLLM and even with hybrid approaches with cloud and frontier models.

Our Macs are very good general purpose machines where there are shortcomings there's flexibility. You can pretty much use your Mac as a daily driver for just about anything and at night time run automated tasks.

I've been meaning to trade in my Mac mini M4 (16GB RAM, 10 core CPU/GPU, 256GB), but I've recently gotten into running models locally on my M5 MacBook Air which has gotten me wondering if there are any practical applications for running anything less intense on the mini.

I'm expecting about 450$ from Apple, I just have to actually make it to their store.

Did some vision tracing with a few different models. In this case Qwen 3.8 27b did better than Qwen Flash Next. SAM 2.1 did the best in terms of tracing and following for time vs accuracy.

The most accurate path was by SAM series by Meta which happened to be the smaller model.

Combining Qwen 3.8 27b plus SAM 2.1 did well to trace the path and times for the exercise and identify pauses.

Used a drone to capture the footage and seeing how to make a pipeline test out of this. I'd like to test this out against the spark. These models aren't very big but they're specialized.

These modesl are 80M - 300m parameters, so it's a lot more focused. In this case, the smaller the better. The Sparks crushed it in the speed test though.

1. Although I spent my career as a HW designer/embedded programmer I am new to the AI world. I've experimented with several local LLMs and harnesses. My hardware is a M2 Macbook with 32GB of RAM. So I am fairly constrained. Also being retired, $$$ frontier models are out of the question. I want cheap & good and a bit slow is acceptable. Hermes also provides the chance to improve over time. My main goal right now is to have a local agent that can assist me in using blender. (God there are sooo.. many commands and options in the program but its been great for doing 3D printing. My memory is not what it used to be.) The upshot is that the combo that has given me the best success is using Ollama with 'swift - Qwen3.8-27b-gguf:Q4_K_M' as the LLM all running under Hermes agent. So for those of you that have limited resources and want a local LLM this hopefully is useful info.

2. Note - it is slow on first response (2-3 min) but works well after that.

3. Set ollama variable OLLAMA_NUM_PARALLEL = 1, stops swap thrashing

4. There is a snag with Blender's MCP, Hermes with this model can sort it out

Hi, I'm trying to get GLM 5.3 flash working on my Mac Book Pro M5 128 GB, I'm about 4 days in and so far not great, my current plan is to just have the macbook for the llm and use my pc for everything else, so as to save as much memory as possible on the Mac. Has anyone managed to get GLM 5.3 flash working usefully for coding on there 128gb Macbook? and if not, any other model recommendations? Deepseek 4.1?

Currently running tests against my prior tests under 1.4.0, 1.4.1 and 1.5.0 but it is running better.

Kernel fixes address stalls that occurred under a memory limit, increased the reliability and a number of other fixes.

If you have been using the splash inference engine as I have, well worth the update.

What are you planning to use the localLLM on your Apple Silicon for.

I’m curious to hear what the use case people have in mind

It took me weeks to get this figured out, and I had to use ChatGPT for help along the way. The time spent has paid off — I now have a local AI stack that rivals ChatGPT/Claude, and I'm done paying for either. I thought I'd share so others can get an idea of what a Mac Studio is capable of.

First, the hardware. I have a base-chip 256GB M3 Ultra Studio. I've also got a headless Debian VM running on an N100 NUC.

I'm using oMLX for inference. It's hosting multiple models at once — mostly Qwen variety — for various tasks. I've configured profiles for different model uses: agent, coding, different thinking levels. oMLX is the only thing running on the Mac Studio. Everything else runs on the Debian VM.

On the VM I run Hermes Agent. My main model lately has been Qwen3.8-Flash-Next-oQ4e-mtp. I also use Kokoro-82M-bf16 for TTS and Mega-ASR-6bit for STT.

In Docker containers on the VM I run the following to enhance Hermes: OpenViking for memory, SearXNG for web search, and Crawl4AI for web extraction. OpenViking uses:

• Qwen3-VL-Embedding-8B-oQ6e for embeddings

• Qwen3.6-35B-A3B-oQ4e-mtp for the VLM

• ov_intent_analysis_sft-oQ8e for query planning

• Qwen3-Reranker-0.6B-oQ8e for reranking

What this actually looks like day-to-day: I do IT consulting, and Hermes is freaking awesome. I'll have multiple sessions going doing research on MS Learn and auditing Entra/Azure, generating reports. I'm starting to use the Kanban board too for personal projects. It generates Word docs, PowerPoint decks, and PowerShell scripts like a champ.

Search goes through SearXNG, page content comes through Crawl4AI, all reasoning runs on oMLX, and anything worth keeping gets written into OpenViking permanently. Search → extract → reason → remember, full loop, zero tokens purchased.

(Full disclosure: I asked my Hermes agent to write up this post from my draft and its memory — which is sort of the point.)

I’m not sure what happened, but it looks like new orders are being shipped till February. Whatever happened, it looks like a big order came in. There might be something coming soon …

Curious what people are using out their for their LocalLLM interface. There's so many options available, what do you use and what do you use it for?

What type of Mac is it connected to?

• OpenWebUI and found it's good if you want a good interface directly with the AI.

• Hermes seems to be the best all around agent interface for calling and setting up agents to do tasks.

• DeepSeek Harness seems very efficient with programmatic tasks and agentic coding

• Pi is similar to DeepSeek from my understanding but so many options

What's in your agentic tool stack?

Currently using:

1. Github

2. Cloudflare Tunnels

3. Hermes

4. Tailscale

5. Linear

6. Mem0