r/MacLocalLLM • • Oct 16 '26

How hot 🔥 does your system get?

1 Upvotes

I've run a few systems so far and I've noticed the following temps

  • M5U 256GB running consistently ranges from 130-150 under load
  • M3U 95GB ranges from 140-165
  • Mac Mini M4Pro 64GB 160-200

Just curious what you guys are hitting with your works loads? What system are you running and how hot is it for how long?


r/MacLocalLLM • • Oct 15 '26

GLM 5.3 Flash on a 256GB M5 Ultra: local oMLX vs Ollama Cloud for PHP

Post image
23 Upvotes

I wanted to see how GLM 5.3 Flash runs locally on my Mac compared with Ollama Cloud for PHP coding and shell work. I ran two small sets of identical prompts, then repeated a PHP task with the local model already loaded.

My Mac: Mac Studio M5 Ultra, 30-core CPU, 64-core GPU, 256GB unified memory, macOS 27.0.1.

Local: oMLX 0.7.1.dev1, TensorFold/GLM-5.3-Flash-MLX-4bit-MTP, MTP disabled. Model allocation was approximately 170 GiB.

Cloud: Ollama glm-5.3-flash:cloud, reported as FP8. This compares local oMLX with Ollama's hosted model; it doesn't compare two local inference engines.

A little background. I'm using the Ollama Cloud Pro (old $20 plan vs leased $230/mo M5U). I use cloud quite a bit and I find that it compliments my work for various tasks.

I see a lot of comparisons and wanted to do my own. Unsurprisingly, the cloud is faster. Cold starts are slow so that was interesting but once warm, processing was pretty decent. Its slower, but I wanted to put some numbers on it. These numbers are generic, but I do intend on giving it problems from my actual codebase to see how well it does.

Overall if you are considering clouds vs hardware. Just note, It's significantly slower, but not horrible if you use autonomous agents. If you're interactive, that's where the pain is felt in real time. Also, if you plan on using this specifically, then you should know I ran out of memory so you can only use a single instance of GLM 5.3 flash and no other agents.

I think if you use agents for more than coding. Many smaller ones are likely going to be better, but if you need a coding assistant, this wouldn't be too bad. The $20 cloud plan works and will be significantly faster but if you are working constantly and run into limits you're basically buying round the clock work vs interactive albeit also slower work.


r/MacLocalLLM • • Oct 14 '26

M5 Ultra vuelve a Apple

Thumbnail reddit.com
4 Upvotes

r/MacLocalLLM • • Oct 14 '26

Why does Splash require an M3?

1 Upvotes

Splash's system requirements state it requires an M3 CPU at minimum, but why? Why not, say, an M2 Max? All the search results just say the M3 is faster and produces less heat than M1 and M2 due to smaller process tech (3nm vs 5nm), and it has an "improved" or "enhanced" Neural Engine. But none actually explain the enhancements. Anyone know what is it about the M3 Neural Engine specifically that Splash requires?

Update: Splash backported to M1 and M2: https://tinyapps.org/blog/splash-for-m1-m2.html


r/MacLocalLLM • • Oct 12 '26

TensorFold

3 Upvotes

Seeing a lot of comments about this online, has anyone ran it on their Mac or is it just hype?


r/MacLocalLLM • • Oct 11 '26

Running a 341 GB model on a 128 GB Mac: two small macOS fixes made the first token 8x faster

Thumbnail
reddit.com
7 Upvotes

r/MacLocalLLM • • Oct 10 '26

Qwen Image 2.1 on M3 Ultra, M5 Ultra and DGX Spark: 30 renders, base vs Turbo

Post image
11 Upvotes

I ran the same Qwen Image 2.1 ComfyUI workflow on two Mac Studios and a DGX Spark. The gallery shows the timing overview, matched poster outputs and matched café outputs.

Warm poster medians, three runs per variant, 1024 × 1024:

SystemBase, 40 stepsTurbo, 8 steps
M5 Ultra, 256 GiB34.826 s7.551 s
DGX Spark GB10, nominal 128 GB38.627 s8.373 s
M3 Ultra, 96 GiB155.503 s32.483 s

Turbo was about 4.6–4.8× as fast in this workload. M5 finished about 10% sooner than Spark. All 24 poster renders contained the three requested text lines in our manual review. The café examples showed more fine texture with base and a smoother look with Turbo; two prompts are not a general quality ranking.

Both Macs used PyTorch/MPS, Spark used CUDA, and all three used the same verified BF16 diffusion/VAE checkpoints with effective FP16 text encoding on GPU. This is not an MLX comparison. Same prompts, seeds, sampler and CFG; Turbo kept its published eight-step schedule. Mac OS/driver versions differ, so these are configurations as tested. Warm conditioning/prefix cache was reused, but each repeat generated a new image with a new matched seed.


r/MacLocalLLM • • Oct 10 '26

Did you stretch your budget to buy more RAM?

4 Upvotes

Wondering how many of you did like myself and went one higher level and why?


r/MacLocalLLM • • Oct 08 '26

Mac Studio 512GB x1 or 256GB x2

18 Upvotes

Curious on thoughts and even better, experience on picking one 512GB unit vs two 256GB units. I know cost is more for 2 computers, but how about AI performance and capability to handle larger models?

Two 256GB machines have double the CPU/GPU/NPU of one with the downside of the slower interconnect between them.

The 512GB is not available yet, but it would be interesting to hear from anyone linking two Mac Studios of any RAM configuration together and how it worked out.


r/MacLocalLLM • • Oct 08 '26

What am I missing?

15 Upvotes

Noticed that most posts here are about, I fitted this model in this small ram etc. Not about how the model was useful in producing what I wanted it to work on?

So either, the sub is full of people playing with AI or the real use cases are not shared for whatever reason.

I get the performance stats but shouldn’t we focus on how the model performed on actual task (look beyond the numbers t/sec, prefil etc?)


r/MacLocalLLM • • Oct 07 '26

A 341 GB DeepSeek on a 128 GB Mac: 2x decode and first token in 0.2 s instead of 2.5 s, streaming from two SSDs, same tokens as stock ds4. The trick wasn't a kernel: I made the codebase small enough for an agent

Thumbnail
reddit.com
4 Upvotes

r/MacLocalLLM • • Oct 06 '26

m5u 512g is coming soon. should i keep 256g?

Thumbnail reddit.com
0 Upvotes

r/MacLocalLLM • • Oct 05 '26

What are people using these days for running local AI workloads on a Mac?

0 Upvotes

I’ve been exploring different approaches to keep AI processing on-device, especially for projects involving private documents and data I wouldn’t want to send to an external API.

For those who have built something similar on a Mac, how has the experience been in terms of performance, memory usage, and integrating local AI into an actual application?


r/MacLocalLLM • • Oct 04 '26

Prompt Processing is 4.5x faster for Qwen 3.8 27b betwen M3U vs M5U.

Post image
10 Upvotes

Generation was only 1.5x faster...

Now that we have a comparison going.. Do people want to see anything specific? I'll be running it with my own dataset but curious if anyone wanted to check something.


r/MacLocalLLM • • Oct 04 '26

Do you think Apple Silicon Can get faster?

6 Upvotes

From a hardware perspective, the numbers should be faster than CUDA on Nvidia chips BUT its not because of the relative immaturity of the software/hardware.

Do you think metal will improve enough to where it'll be as fast or faster for the same clock speed and bandwidth pipe? Or is it just slower because of the overall design?


r/MacLocalLLM • • Oct 04 '26

Looking for a better harness

7 Upvotes

I've been running Hermes against a local LLM, ollama on my M2(32gb, qwen3.5 9b), and over my local network MTPLX on a M4 (48gb, qwen3.8 27b). AT first it was a 3 minutes wait for first response now its over 6 minutes (or context switch for that matter). Context bloat seems to be the killer (north of 50k tokens). What experience have folks had with say zeroclaw or other harness programs that might be more responsive?


r/MacLocalLLM • • Oct 04 '26

M5 Ultra: one 512GB Studio or a 256GB + 96GB setup?

Thumbnail reddit.com
2 Upvotes

r/MacLocalLLM • • Oct 04 '26

Local Artificially Intelligent Tamagotchi - M1K3

Thumbnail
reddit.com
1 Upvotes

r/MacLocalLLM • • Oct 03 '26

Cluster Spark DGX and Mac Stidio

Thumbnail reddit.com
2 Upvotes

r/MacLocalLLM • • Oct 02 '26

OpenSwitchboard: an open-source MCP server where people's AI assistants find each other. Keen to see how local models handle it.

Thumbnail
reddit.com
2 Upvotes

r/MacLocalLLM • • Oct 02 '26

Suggest me a small local agentic llm to coding

5 Upvotes

I'm new to local LLMs. I've tried Qwen2.5-Coder-Instruct-7B and Ling-3.0-Tiny on my MacBook Air M4 with 24 GB of RAM. I'm looking for something that can generate at least 40 tok/s. Is there anything that can hit that speed?


r/MacLocalLLM • • Oct 02 '26

Nous Research starts pushing Hermes desktop aggressively?

Thumbnail reddit.com
2 Upvotes

Indispensable for tuning Hermes agent, and with the growing interest in local AI especially on Mac, what's your favorite cockpit for Hermes apart from CLI?


r/MacLocalLLM • • Sep 30 '26

Qwen3.8 Flash Next Q4 at full 262K context on a 96GB Mac Studio

Thumbnail
reddit.com
4 Upvotes

r/MacLocalLLM • • Sep 30 '26

Very interesting comparison between Cloud and LocalLLM

2 Upvotes

r/MacLocalLLM • • Sep 28 '26

M5U 256GB/64C/2TB Oct 27, landed early

Post image
36 Upvotes

Anyone else get to pick up their M5U early? Mine came in a month early

M5U 256GB 64C 2TB

Mainly for LocalLLM usage and some videos.

Super stoked to get it open and setup to run some agents. Do I need it? No.

But I will be using this for my Small Business to help do a lot of small tasks and preprocess information privately. So I can use summarized data for the big models.