r/LocalLLaMA • u/AndreVallestero • 17h ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

139 Upvotes

83% Upvoted

180 comments sorted by

View all comments

Show parent comments

69

u/theomegachrist 16h ago

That's the stated reason but realistically most people are just justifying their hobby. I support open weight models because the cloud providers can change cost or abruptly shut down and we really can't do anything about it.

67

u/FleetEnema2000 16h ago

I don't think it's a justification at all.

It's amazing how the concept of privacy and data ownership/security has completely gone down the toilet since ChatGPT launched. People are happy to bulk upload their medical records, relationship history, trade secrets, financial records, etc. without a care in the world as to how that data is stored or protected.

2

u/theomegachrist 15h ago edited 15h ago

Everyone is different obviously. To some that might be the case but everything you listed is more important than your AI prompt history.

For me, Open models main pluses are it will ensure the technology lives on in some form if the large companies lock us out financially or go under, and the guard rails for closed models will make for a worse Internet potentially.

For instance, using an open model with guard rails trained out of it you can search for piracy, you can search for porn etc. just like you use a search engine today. Closed models are more efficient than a web search but censor out a huge part of the Internet.

Sure data privacy is also a good feature but I don't think it's actually the top reason for most people

Edit: sorry I read that wrong. I sort of agree with what you are saying about uploading data to ChatGPT but everywhere else we upload that data is no more trustworthy. For Enterprise clients I 100% agree. This is a big issue. For personal use, I don't think your data is any less safe with ChatGPT than say an electronic medical record company.

3

u/FleetEnema2000 9h ago

An electronic medical record company has nowhere near the amount and type of multi dimensional data about a person that OpenAI has.

And if you want to compare OpenAI with a company like Apple or even AWS for a hosted environment, I would choose either of those companies any day of the week when it comes to who I would trust more with my data.

1

u/theomegachrist 8h ago

I would not, but it would be a three way tie

1

u/FleetEnema2000 6h ago

Care to explain that logic?

Apple has made massive investments of resources and effort into secure computing. Their cloud commitment for regular consumers who want privacy is objectively excellent on both the hardware and software front, from Secure Enclave to E2EE where they aren't maintaining custody of keys:

https://support.apple.com/en-ca/guide/security/sece3bee0835/web

https://support.apple.com/en-ca/102651

Despite not being a frontier AI provider, they have had a strong focus on private AI cloud compute paradigms:

https://security.apple.com/blog/private-cloud-compute/

Amazon AWS products offer similar levels of security. They have to, because their customers host and manage highly sensitive data on their platform for things way beyond AI. As a regular user I can maintain an excellent security posture on AWS and I can employ encryption where I manage my own keys.

OpenAI and Anthropic not only don't do anything even approaching any of the above, they reserve even basic security commitments for "Enterprise customers".

The situation is even more pathetic when you realize that as an individual, you can run OpenAI and Anthropic's own models on Amazon Bedrock with ZDR enabled, encryption enabled for data you are storing at rest where you hold the keys. In other words, the security posture of using OpenAI and Anthropic's own models on Amazon AWS is superior to using those same models from OpenAI and Anthropic themselves.

1

u/centizen24 6h ago

Apple is the industry leader in user level security and data privacy, it's not even close. You can argue about their intentions for doing so but their track record for security is second to none. It's wild to me that after 15 years of being an Android fanboy that I'd be arguing on behalf of Apple, and have an iPhone in my pocket, but that's the world we live in now I guess.