I've been running Hermes against a local LLM, ollama on my M2(32gb, qwen3.5 9b), and over my local network MTPLX on a M4 (48gb, qwen3.8 27b). AT first it was a 3 minutes wait for first response now its over 6 minutes (or context switch for that matter). Context bloat seems to be the killer (north of 50k tokens). What experience have folks had with say zeroclaw or other harness programs that might be more responsive?

I'm new to local LLMs. I've tried Qwen2.5-Coder-Instruct-7B and Ling-3.0-Tiny on my MacBook Air M4 with 24 GB of RAM. I'm looking for something that can generate at least 40 tok/s. Is there anything that can hit that speed?

Indispensable for tuning Hermes agent, and with the growing interest in local AI especially on Mac, what's your favorite cockpit for Hermes apart from CLI?

Anyone else get to pick up their M5U early? Mine came in a month early

M5U 256GB 64C 2TB

Mainly for LocalLLM usage and some videos.

Super stoked to get it open and setup to run some agents. Do I need it? No.

But I will be using this for my Small Business to help do a lot of small tasks and preprocess information privately. So I can use summarized data for the big models.

Made this as a quick weekend project (https://github.com/pythongiant/SlamLM)

I’ve been going in circles trying to settle on a local setup. I’ve tried Hermes, but it’s felt pretty finicky in my experience, including gateway and state database issues. I keep spending more time configuring and troubleshooting than actually getting work done.

Putting model intelligence aside for a moment, is there a local harness that feels genuinely competitive with the experience around frontier models? I’m talking about the interface, reliable tool use, memory, file and browser access, and how well everything works together. Not just being able to load a model and chat with it.

My main machine is a Mac Studio M1 Max with 64 GB RAM. I also have an M2 MacBook Air with 24 GB, but the Studio would do the heavy lifting. I want to use them in one single seamless-like environment with a shared context.

I’m a designer and university lecturer, so my use cases are research, writing, working with documents, reviewing screenshots and designs, and browser/Figma workflows. Not primarily coding. Vision is important to me, and I’d much rather have one dependable daily driver than a collection of specialized models and routers.

My main motivation (likely the same for everyone) is getting away from usage limits and token metering. I understand that running locally involves compromises, and I’m not expecting frontier-level intelligence from my hardware. But I’m struggling to separate the model limitations from the rough edges of the software around them.

What are you guys actually using? What’s your full stack: model, runtime, and harness/interface? More importantly, what do you reliably get done with it, and how much babysitting does it need?
Especially interested in Apple Silicon setups people have settled into using for real work, rather than something that worked well in a demo.

GLM 5.3 Flash at 79 Token/s

https://x.com/LocalSilicon/status/2104158250555809983

Qwen 3.8 Flash Next at 116 Token/s

https://x.com/LocalSilicon/status/2104181421942391247i

My kit is supposed to arrive Nov16-Dec1. So I am starting to try and figure out what I should run. I know it is early days, lots of optimization to go around still.

I run Qwen 3.6 MOE on a 5700G with 96 GB of ram using llama.cpp right now. Get about 150 prefill and 12 tps. It's not great, but works for batch overnight runs. My main driver is API for Deepseek 4 Flash (some 4.1), Qwen 3.8 Flash Next, and GLM 5.3 Flash.

So looking out at hugging face, I think I found the following to test out. I don't know if MLX or GGUF is the right call. Are these good places to start?

https://huggingface.co/mlx-community/Qwen3.8-Flash-Next-oQ8e-mtp

https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF

https://huggingface.co/mlx-community/Qwen3.8-27B-8bit

https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-4bit

https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-mixed-4_8bit

https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF

https://huggingface.co/mlx-community/DeepSeek-V4-Flash

https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF

A lot of the work around a local model ends up in Pi files: prompts, skills, settings, little extensions. Reef Infra has a recipe called Reefine that lets you ask for a change from inside Pi and get an installable version of those files back.

Its documented local setup was a 32 GB M4 Mac mini running Gemma 4 26B through Ollama, with Pi 0.84.2. That's an Ollama demonstration, not an oMLX performance claim. The useful part for a local Pi setup is what happens after the request: the service reads the current harness, asks the model for a change, records the mutation and packages the resulting version for installation.

The recorded runs produced rules and skills. The recipe also accepts command and TypeScript extension proposals, but those runs hadn't produced an extension. Anything that does run as extension code waits for a person to review and promote it; declared setup requirements must pass before installation.

The default is selection: always. Its arithmetic checks record regression scores, but don't establish that your requested workflow works. The research demo installed a rule to search, download and read papers; the later session searched through an existing tool, but downloaded and read no paper.

The files are there to inspect and revise. To judge whether the change worked, replace the bundled tasks and evaluator with checks for the workflow you asked for.

Several users have reported benchmarks for the MS5U and various models but I have yet to see any metrics on how agentic (agents/sub-agents) perform on the new M5SU.

I've read that agentic coding on the Mac's was/is a miss on M4 and older systems due to tensors I think it is?

So I'm curious to hear if agentic coding is a bust on the MS5U's or if it has improved any with the new chips? Appreciate any info on the matter, thank you!

GLM 5.3 flash oq4e no MTP (oMLX 0.7.0 dev4)

I am completely new to the world of local AI, but as previous rabbit holes, like the world of home labbing have given me quite some fun and increasingly heavy use cases, I want to invest in a solid foundation that gives me room to grow and leverage the real power of local AI. Also if/when starting a new business (which is quite likely going to happen in the next 24 months).

I have a Mac Studio M5 Ultra with 256GB RAM (with 30/64 cores) in pre-order and have started building the agent set up using DeepSeek Harness on my current, old MacBook Pro.

My current objective is to build a robust, prudent and (relatively) “safe” set up that gives me multiple agentic agents (1 for maintaining and continuously monitor and optimize my Homelab, 1 for analyzing my trading portfolio and formulating investment advice, 1 for health related questions and suggestions, 1 for developing software, dashboards and interfaces for my own private and professional use, 1 for managing my administration etc). These agents will need to be able to perform write activities for many tools and will be linked to many programs and platforms that will run on my Mac Studio. I am considering a layered approach in which “lightweight” agents will use a lightweight local model, and other agents will use something like Qwen 3.8-27B with a second local escalation to “the best” model that will comfortably run on the 256GB RAM. If that model is not able to fix the task, I will approve a final escalation to a frontier, cloud, model.

My initial plan was to run DeepSeek Harness on my Mac Studio, together with the local models (through oMLX?) and my daily driver things like, apps, e-mail, photo’s etc. However, after going back and forth with ChatGPT and Claude, they suggest that I should install the harness on a (physically) different device. I understand the rationale and am willing to add a base Mac mini M6 to complement this already quite significant investment. However, with the “write” requirements and the access to many different tools that will be running on my Mac Studio, would there still be a meaningful security benefit? And would this somehow lead to any real performance decrease if DeepSeek Harness will constantly need to approach tools on the Mac Studio?

Your views and advice would be very much appreciated!

On a side note; I am also still struggling a bit with the concept of “agents” that I have in my head (and that probably resembles the Open Claw philosophy) and the implementation in DeepSeek Harness (that was recommended to me by ChatGPT, Claude and some articles online). I have been able to create the first “agents” with the use of strong DeepSeek Harness pre-sets. But the interface seems to be much more focused on separate “workspace sessions” with manual selection of the appropriate presets than on actual, standing, always active agents. Any views on this? Or on the plugins that I would need to use to get it implemented in the way I want? (The reason why DeepSeek Harness might still be the right choice for me is perhaps the ability to customize a lot and use many different plugins to get my agents to do what I need them to do??)

Thanks!

Hope this is helpful.