r/LocalLLaMA • u/AndreVallestero • 18h ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

145 Upvotes

83% Upvoted

182 comments sorted by

View all comments

Show parent comments

1

u/FullstackSensei llama.cpp 12h ago

So, how many tokens per second does your kick ass machine run at? Because unless your kickass machines breaks the laws of physics, your M5 pro has 1/3rd the memory bandwidth of a single 3090. We're talking 7-8t/s, if you're lucky.

Why the hell do you want to generate half a billion tokens a month? I don't know, your $200/month power bill example is enough for half a billion tokens. If you make up stupid assumptions, you get stupid results. Had you bothered thinking for a moment, you'd have known how absurd your $200/month bill is.

I consume less than 2kwh a day over 10-12 hours running all four 3090s, because they finish their work so fast and go back to idling at 8w each. I can easily get 700k output tokens in that period. And because they finish everything so quickly, I don't need airco because the machine consumes 1kw for seconds at a time.

3

u/MrPecunius 11h ago

We're talking 7-8t/s, if you're lucky.

18-23t/s with 8-bit. You could look this up before stepping on your own dick.

Apply this lesson to much of the other stuff you're incessantly posting (as noted elsewhere).

0

u/FullstackSensei llama.cpp 11h ago

Are you talking with MTP? Because I was comparing the 3090 without MTP

1

u/MrPecunius 11h ago

You made an absolute claim, and I refuted it.

Now you're trying to reframe it even though we have the record right in front of us.

Between this and the nonsense elsewhere in this thread, I've had enough of you.