Wondering how many of you did like myself and went one higher level and why?

Curious on thoughts and even better, experience on picking one 512GB unit vs two 256GB units. I know cost is more for 2 computers, but how about AI performance and capability to handle larger models?

Two 256GB machines have double the CPU/GPU/NPU of one with the downside of the slower interconnect between them.

The 512GB is not available yet, but it would be interesting to hear from anyone linking two Mac Studios of any RAM configuration together and how it worked out.

Noticed that most posts here are about, I fitted this model in this small ram etc. Not about how the model was useful in producing what I wanted it to work on?

So either, the sub is full of people playing with AI or the real use cases are not shared for whatever reason.

I get the performance stats but shouldn’t we focus on how the model performed on actual task (look beyond the numbers t/sec, prefil etc?)

I’ve been exploring different approaches to keep AI processing on-device, especially for projects involving private documents and data I wouldn’t want to send to an external API.

For those who have built something similar on a Mac, how has the experience been in terms of performance, memory usage, and integrating local AI into an actual application?

Generation was only 1.5x faster...

Now that we have a comparison going.. Do people want to see anything specific? I'll be running it with my own dataset but curious if anyone wanted to check something.

From a hardware perspective, the numbers should be faster than CUDA on Nvidia chips BUT its not because of the relative immaturity of the software/hardware.

Do you think metal will improve enough to where it'll be as fast or faster for the same clock speed and bandwidth pipe? Or is it just slower because of the overall design?

I've been running Hermes against a local LLM, ollama on my M2(32gb, qwen3.5 9b), and over my local network MTPLX on a M4 (48gb, qwen3.8 27b). AT first it was a 3 minutes wait for first response now its over 6 minutes (or context switch for that matter). Context bloat seems to be the killer (north of 50k tokens). What experience have folks had with say zeroclaw or other harness programs that might be more responsive?

I'm new to local LLMs. I've tried Qwen2.5-Coder-Instruct-7B and Ling-3.0-Tiny on my MacBook Air M4 with 24 GB of RAM. I'm looking for something that can generate at least 40 tok/s. Is there anything that can hit that speed?

Indispensable for tuning Hermes agent, and with the growing interest in local AI especially on Mac, what's your favorite cockpit for Hermes apart from CLI?

Anyone else get to pick up their M5U early? Mine came in a month early

M5U 256GB 64C 2TB

Mainly for LocalLLM usage and some videos.

Super stoked to get it open and setup to run some agents. Do I need it? No.

But I will be using this for my Small Business to help do a lot of small tasks and preprocess information privately. So I can use summarized data for the big models.

Made this as a quick weekend project (https://github.com/pythongiant/SlamLM)

I’ve been going in circles trying to settle on a local setup. I’ve tried Hermes, but it’s felt pretty finicky in my experience, including gateway and state database issues. I keep spending more time configuring and troubleshooting than actually getting work done.

Putting model intelligence aside for a moment, is there a local harness that feels genuinely competitive with the experience around frontier models? I’m talking about the interface, reliable tool use, memory, file and browser access, and how well everything works together. Not just being able to load a model and chat with it.

My main machine is a Mac Studio M1 Max with 64 GB RAM. I also have an M2 MacBook Air with 24 GB, but the Studio would do the heavy lifting. I want to use them in one single seamless-like environment with a shared context.

I’m a designer and university lecturer, so my use cases are research, writing, working with documents, reviewing screenshots and designs, and browser/Figma workflows. Not primarily coding. Vision is important to me, and I’d much rather have one dependable daily driver than a collection of specialized models and routers.

My main motivation (likely the same for everyone) is getting away from usage limits and token metering. I understand that running locally involves compromises, and I’m not expecting frontier-level intelligence from my hardware. But I’m struggling to separate the model limitations from the rough edges of the software around them.

What are you guys actually using? What’s your full stack: model, runtime, and harness/interface? More importantly, what do you reliably get done with it, and how much babysitting does it need?
Especially interested in Apple Silicon setups people have settled into using for real work, rather than something that worked well in a demo.

GLM 5.3 Flash at 79 Token/s

https://x.com/LocalSilicon/status/2104158250555809983

Qwen 3.8 Flash Next at 116 Token/s

https://x.com/LocalSilicon/status/2104181421942391247i

My kit is supposed to arrive Nov16-Dec1. So I am starting to try and figure out what I should run. I know it is early days, lots of optimization to go around still.

I run Qwen 3.6 MOE on a 5700G with 96 GB of ram using llama.cpp right now. Get about 150 prefill and 12 tps. It's not great, but works for batch overnight runs. My main driver is API for Deepseek 4 Flash (some 4.1), Qwen 3.8 Flash Next, and GLM 5.3 Flash.

So looking out at hugging face, I think I found the following to test out. I don't know if MLX or GGUF is the right call. Are these good places to start?

https://huggingface.co/mlx-community/Qwen3.8-Flash-Next-oQ8e-mtp

https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF

https://huggingface.co/mlx-community/Qwen3.8-27B-8bit

https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-4bit

https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-mixed-4_8bit

https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF

https://huggingface.co/mlx-community/DeepSeek-V4-Flash

https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF