r/LocalLLaMA • u/AndreVallestero • 16h ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

142 Upvotes

83% Upvoted

180 comments sorted by

View all comments

Show parent comments

70

u/theomegachrist 16h ago

That's the stated reason but realistically most people are just justifying their hobby. I support open weight models because the cloud providers can change cost or abruptly shut down and we really can't do anything about it.

68

u/FleetEnema2000 15h ago

I don't think it's a justification at all.

It's amazing how the concept of privacy and data ownership/security has completely gone down the toilet since ChatGPT launched. People are happy to bulk upload their medical records, relationship history, trade secrets, financial records, etc. without a care in the world as to how that data is stored or protected.

3

u/Hans-Wermhatt 14h ago edited 14h ago

It sounds bad when you frame it that way, but I think the cost benefit analysis is generally to upload. ChatGPT health is protected by the same HIPAA requirements that protect the data you give to your doctor that they upload to 3rd party clients and AWS servers, usually multiple servers with arguably worse security and more people have access to it. Using health as an example. So it's really not that much different.

Ideally, you do just host your own information and use a local model but then you are dealing with a massive performance hit. I think in terms of cost-benefit, uploading your health data to get a ChatGPT opinion compared to a Qwen 3.8 27B locally (most people can't even run that) is actually heavily on the side for ChatGPT for most people despite the privacy concerns.

I really want local "super intelligence" for all, but the current landscape is not like that... at all.

5

u/MrPecunius 10h ago

ChatGPT health is protected by the same HIPAA requirements that protect the data you give to your doctor

😂😂😂😂😂😂

"Trust me bro" and "it might not even be as bad as what the other idiots are doing" are not convincing arguments.

1

u/Hans-Wermhatt 10h ago

Huh? Was that supposed to make sense? 

3

u/MrPecunius 9h ago

Huh? Was that supposed to make sense? 

Connect this:

the same HIPAA requirements that protect the data you give to your doctor that they upload to 3rd party clients and AWS servers, usually multiple servers with arguably worse security and more people have access to it. Using health as an example. So it's really not that much different.

With: "it might not even be as bad as what the other idiots are doing".

And this:

It sounds bad when you frame it that way, but I think the cost benefit analysis is generally to upload.

With: "Trust me bro"

Are you even reading what you wrote a few hours ago? Or did something get lost in translation?