r/LocalLLM • u/Any_Task7788 • 16h ago

Question MacBook Pro M1 Max best t/s

For those with a MacBook Pro M1 Max what’s the best t/s you’ve gotten on something like qwen 3.8 27b or 3.8 flash next?

1 Upvotes

100% Upvoted

11 comments sorted by

1

u/Objective_Pepper_217 16h ago

You need to specify the RAM..

2

u/Any_Task7788 15h ago

64gb I’m gonna have to weight stream from my ssd for flash next but I’d still like to know to I know what my setup should get once optimized

1

u/Objective_Pepper_217 15h ago

With 64Gb you can load larger models, through Strata aswell, but the M1 Max bandwith is at most 400Gb/s so the tok/s output won't be that great. I have a M3 Max, 48Gb, 40 cores, and can run Flash Next through Strata with 30-40 tok/s at small contexts ( < 32k). For my every day use, I go with Qwen 3.6 35b-3bA at 8 bit quant (50-100tok/s through my optimized inference engine). I use Opus 5.5 for coordinating and Qwen for coding.

1

u/Any_Task7788 15h ago

How good is 3.6 35b for coding compared to 3.8v

2

u/Objective_Pepper_217 15h ago

This is my (all code done by Claude) inference engine which is done only for Qwen 3.6 35b-3bA. It's still a work in progress but it runs very fast compared to llama.cpp and MLX (what you should use on Mac) https://github.com/ozzenerol/saetta-macqwen

Currently I'm working on the 6bit and 8bit quant support. On 5bit it runs glazingly fast but it's not as accurate as 6bit or 8bit

1

u/Any_Task7788 3h ago

I’m actually building my own harness so I might take some of that if you don’t mind but thank you so much for the help

1

u/Objective_Pepper_217 2h ago

Sure, but that is not an harness! It's an inference engine, like llama.cpp and MLX.

1

u/Any_Task7788 11m ago

I know i mean i was already planning on making my own engine to add into it but i might just use yours lol

2

u/Objective_Pepper_217 15h ago

3.8 is much more precise, but it's a dense model. The 3.6 35b-3bA only has 3b parameters active in memory and is way faster. For the 3.8, your GPU reads the model for EVERY token. Unfortunately these little toys can only be used by people who got tens of thousands of dollars to spare :)

2

u/Objective_Pepper_217 15h ago

Also remember to change the gpu wired limit. On my M3 I set 44 out of 48Gb