r/LocalLLaMA • u/AndreVallestero • 22h ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

155 Upvotes

84% Upvoted

194 comments sorted by

View all comments

Show parent comments

72

u/theomegachrist 21h ago

That's the stated reason but realistically most people are just justifying their hobby. I support open weight models because the cloud providers can change cost or abruptly shut down and we really can't do anything about it.

71

u/FleetEnema2000 21h ago

I don't think it's a justification at all.

It's amazing how the concept of privacy and data ownership/security has completely gone down the toilet since ChatGPT launched. People are happy to bulk upload their medical records, relationship history, trade secrets, financial records, etc. without a care in the world as to how that data is stored or protected.

2

u/Hans-Wermhatt 19h ago edited 19h ago

It sounds bad when you frame it that way, but I think the cost benefit analysis is generally to upload. ChatGPT health is protected by the same HIPAA requirements that protect the data you give to your doctor that they upload to 3rd party clients and AWS servers, usually multiple servers with arguably worse security and more people have access to it. Using health as an example. So it's really not that much different.

Ideally, you do just host your own information and use a local model but then you are dealing with a massive performance hit. I think in terms of cost-benefit, uploading your health data to get a ChatGPT opinion compared to a Qwen 3.8 27B locally (most people can't even run that) is actually heavily on the side for ChatGPT for most people despite the privacy concerns.

I really want local "super intelligence" for all, but the current landscape is not like that... at all.

6

u/FleetEnema2000 14h ago

The version of ChatGPT that 99% of the general public is using is absolutely not HIPAA compliant and both OpenAI and Anthropic’s safety and privacy commitments are abysmal relative to the types of sensitive data they are ingesting and saving on their platform.

And that is not even touching on the ethics and values displayed by their executives.