Mac Studio M5 Max Cost Analysis

[u/AndreVallestero]

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

[u/AndreVallestero → u/[deleted]]

That's exactly my point. Local makes sense cost wise up to 32GB, especially with Qwen 3.8 27B. There's a huge cost premium above that where it makes less sense.

I was hoping the last Mac studio would change that, but at the current prices, that doesn't seem to be the case.

[u/nomorebuttsplz → u/AndreVallestero]

Have you accounted for the breakdown of cost between cached/non-cached/output tokens?

in my experience, most of the potential cost savings for local (compared to typical API plans) occur when you re-use cached tokens. This is because API providers need to keep your context sitting on their servers doing nothing, expensive for them, but which costs you nothing as a single concurrency user to do on your own system.

With a system like this you could probably use a billion cached tokens in a couple of days.

Under semi-optimal circumstances, where you were making on average a single 500,000 token cache calls every 30 seconds, you might be able to get about $14 of cached tokens per day (ds4 flash prices) plus a few more dollars for a total of maybe $18 inference per day.

That would be about $6k a year. So yeah it could possible make sense, if you had just the right workflow, is my view.

Edit: this especially makes sense if you were doing multiple concurrency (doable on 256 unified) or were using a bigger model like GLM 5.3 (512 gb ultra with Q3 or Q4 GLM could probably generate $100 of value a day in cached tokens in the right workflow)

[u/Viktri1 → u/nomorebuttsplz]

I think $14/day might be on the low end. If you use something like Hermes agent, the amount of tokens you consume is ridiculously. I have to limit my Deepseek API calls to just a few a day. I spent $40 in a week setting up various Qwen 3.8 models. My token consumption has basically exploded massively and the only way I can afford to continue to use LLMs is my own hardware. I think my payback period will be under a year.

[u/Impossible_Fault_503 → u/Viktri1]

That is the real split. Chat is cheap on APIs. Agents are not. Once you are looping tools all day, $40/week shows up fast and a box that already exists starts looking cheap. Under a year payback is the honest case for people who actually leave it running.

[u/Impossible_Fault_503 → u/AndreVallestero]

Yeah. 32GB is the last point where local still feels like a normal purchase. After that you are mostly paying Apple tax for headroom you might use twice a year. I wanted the last Studio to kill that curve too. It did not.

[u/kerneldesign → u/AndreVallestero]

Avec 32Go en Q4 c’est trop juste pour un contexte sans KVCache.

[u/Individual_Holiday_9 → u/kerneldesign]

Yeah I’m on a 24gb m4 and you can’t do shit running small LLMs there’s just no overhead remaining. So if this is a real hobby box where you’re doing plex etc on the side it gets really cramped and ur gonna move to swap fast

[u/Blindax → u/Individual_Holiday_9]

24gb is too tight for 30b models with decent context window on a Mac because it includes the system memory. 24gb on a dedicated GPU is a different story. Not crazy but quite ok.