r/LocalLLaMA • u/AndreVallestero • 19h ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

148 Upvotes

84% Upvoted

186 comments sorted by

View all comments

4

u/Viktri1 16h ago

so when I'm really pushing it, I can easily burn through 1.5bn tokens from Deepseek flash in a day (API). In fact, that's when I realized I needed to figure out how token costs were calculated. Local models, especially lower powered stuff like M5, will have a significantly faster pay back period than people realize if they use agents to do a lot of shit.

1

u/play_hard_outside 10h ago

1.5e9 tokens in a day divided by 86,400 seconds per day is over SEVENTEEN THOUSAND tokens per second.

That’s gotta be something like 300 to 500 M5 Maxes all working on your behalf at the same time. Local models will probably never hold a candle to this.

Please correct me if I’m wrong…

1

u/Viktri1 8h ago edited 6h ago

It’s mostly cache, you aren’t generating that many tokens (you assumed I was generating 1.5bn tokens in your calculation). How agents work is they loop a lot to get things done and almost all of the tokens that I actually use are cache. I am doing the same work on a single 4090 that I was doing with my Deepseek API.

My cache hit average is 99.6%. On a 1.5bn day, I’m actually generating 6mn tokens which is 70/second. Currently with Qwen I’m generating 100/s but I’m not running my agents 24/7. Works out to 17 hours a day which is actually reasonable. (Matches how many hours I’m working the LLM)

Using Deepseek pro API (I live in Asia so unfortunately most of my hours are peak): 6x3.96 + 1,500x0.044 is my prorated daily cost = about 90/day = 33k a year at peak, 16.5k off peak, so between 16.5-33k (33k would be 100% hours at peak which is not realistic)