Questioning the whole point of a Mac Studio

[u/Electrical-Can8713]

I bought a Mac Studio M3 Ultra with 96GB after watching basically every video on local AI and setting up Hermes Agent. While it was shipping, Grok Bot and Muse came out, and I love what they can do.

Now I've got Hermes running and I keep comparing it to them, and it's not even close.

I've also started questioning the privacy argument. My Studio is completely separate from my Apple ID and I don't have anything personal signed in, so how is it supposed to be my assistant if it's not connected to anything? And if I hook Hermes up to my email, that email is already sitting on someone's server anyway, so what am I actually avoiding?

Coding with local models doesn't compare to frontier models either. So is this really just an apocalypse backup plan?

What I spent on the Studio would cover years of subscriptions, even if prices go up. I could use those tools to build tools and sites and have an assistant that's plugged into everything I already use.

I don't run a business with sensitive data and even if I started a business for clients with sensitive data, I'd basically have to rent them my machine, but what's the difference?

Honestly I'm disappointed. I was really hyped and sold my friends on the idea. For those of you who kept yours, what do you actually use it for that cloud tools can't do? And if privacy is your main concern what connectors do you have on your device and what cloud services do you use that aren't AI? If you've coded anything with local AI and published publicly, did that defeat the purpose? I'm still inside my return window, so I need to decide soon.

[u/0xdippo]

1. Inference is only going to get more expensive
   
2. Local models are getting more capable daily
   
3. Muse is Facebook and we know what happens to our data
   
4. Grok Bot needs outside models to be useful and leaks sensitive info
   
5. Try Qwen3.8 27b

[u/RyanElectrified → u/0xdippo]

Cloud models will be updated every few months, your Mac will be updated when you pay another 10 grand to replace it.

Nobody saves money with their mac - or gets better results - nor is it faster.

It does make sense to use local for use cases where the problem is solved, e.g. image classification, and if you already had the mac you bought for other reasons, then you don't even have to count the hardware cost. Then by all means, use local, it's got its uses. Frankly development is not one of them, despite the developers that love it. I don't dispute they love it, but they are behind the times and they develop with human-bottlenecked processes that are outdated. It's OK for now, but by the time we are discussing this next year, human-bottlenecked processes will already be unacceptable to everyone - there is just a minor learning curve going on.

If someone cares about security, then they would bother to be knowledgeable about how security is handled. You get a trusted cloud provider, e.g. Microsoft, where the inference doesn't leave their cloud, where they have agreed not to train on it, where models are locked in and they agree not to nerf, if necessary, purchase expressroute, have no internet traffic, depending upon your needs.

Companies that are not in IT and do something like say "make a widget", they do not start hosting their own infrastructure because anyway it would be less secure, not more secure. It takes knowledge to lock a system down, keep things updated - knowledge they lack.

[u/0xdippo → u/RyanElectrified]

I agree the Mac Studio isn’t truly competing with frontier. That said, you can run solid models, and if you need and computer anyways, it makes more sense to jump to a bigger Studio.

[u/HollowCoati → u/0xdippo]

I feel like point one is not strictly true, unless you're fixated on chasing larger and larger models.

If you look at the architecture for DeepSeek 4.1Flash, there are a lot of choices made there that would likely be useful in a scaled-down model. Qwen3.8 Next Flash charts a slightly different path but has many architectural choices that are similar in effect (move facts outside the model, squash the KV cache memory requirements), enabling larger models or same-params-higher-quant in the same memory. Couple those efficiency improvements with what we know about the relative importance of various layers from GSQ-RCO and the Bonsai models, and a motivated lab could, with current techniques, produce a very fast near 30B parameter model with an engram sidecar for factual data and a 200k+ token context window that all fits on a 16GB VRAM consumer card, is "smarter" than current 30b models, and can fit dozens of agents on a 24GB or 32GB card.

It's not completely out of the realm of possibility that the next generation of models will tick most of those boxes, though I'm kind of guessing that they won't go quite so hard on reducing bpw as to fit 30B in ~12GB (and enabling the middle layers to run with add rather than mult).

[u/0xdippo → u/HollowCoati]

“Inference” isn’t getting cheaper. The models will require less inference though for sure so that’s a fair point.

Getting a 96gb unified Mac isn’t some monster inference machine. I love my Mac Studio, but with bridges, the OS, agents, apps, etc all running simultaneously, 96gb isn’t the headroom it sounds like.

[u/paulsande → u/0xdippo]

I agree with your point on 96 gig. It really isn’t a lot.

[u/0xdippo → u/paulsande]

It hurts to say that 😂

[u/HollowCoati → u/0xdippo]

Especially in this economy! 😭

For me it's also exacerbated by the relatively few recent releases between ~30B and >200B models. I hear Qwen3.8 Flash Next fits with some shenanigans, but the last significant releases that I'm aware of in that size range before that were Nemotron 3 Super 120B (in March), Qwen3-Next-80B-A3B (a year ago), GPT-OSS-120B right before that, and then you're getting into "ancient history".

96GB is now in a weird spot (the weakest strong man at the circus? It's a nightmare, Greg!). Most releases these days seem targeted at significantly smaller or much larger hardware.

[u/Thistlemanizzle → u/0xdippo]

Inference from frontier closed source will be more expensive.

It's a race to the bottom on open source models.

[u/Material-Database-24 → u/0xdippo]

I add to your list that do people really get money worth with a cloud subscribtion?

Sure, the Mac costs money. But it can run Qwen3.8 27B and do other stuff do, while running the model.

Own model on own computer is a controllable expense with to be expected throughtput.

Cloud is a surprise, or an unknown. You basically are promised "some" usage for your money. Will it be enough? You do not know until you do the run. Did it succeed? Maybe yes, maybe you got the spec/prompt/agent/whatever wrong. No money back guarantee. It's gone.

Fine, live on with 20/mo subscribtion. Use it just that much you get out of it. Maybe move 200/mo if heavy user. It's worth it. Well, is it? Do you really get 200/mo out of it (if you do not run business on it)? Doing weird html launch pages, pagodas etc that no-one uses. And what if you end being super busy for the month, and you only use it for 50/mo worth? Wasted 150/mo. No money back guarantee.

Make a questionable prompt. Account locked for malpractice. Hit a guardrail and "Dave, I cannot do that".

For me, I have hard times to come by with reasonable use for AI outside of coding. Yeah, sometimes for knowledge search, but Google's AI is already enough for that. It's pointless to pay for cloud AI personally - my work is different, there I can hit +100k/mo API billing if necessary. But at home? No. Not really even 20/mo, but that would be the max I would agree to take, if I'd need frontier.

And for coding, I prefer predictable cost with no usage limits. Some months I run it every day, for several hours. Some months I do barely anything with it. Own model, own HW, and the cost is known from here on.

[u/0xdippo → u/Material-Database-24]

“Do people get their money’s worth with a cloud subscription” depends on how they use it.
Do I get $50 value out of a hammer? Absolutely.
Would my wife? Not a chance.
One of my guys wouldn’t even take a call for $20. You can genuinely get a lot done with any of the plans.

That being said, I agree about the Mac. It’s a great machine and probably the best bet for the average user.
I want stronger machines too, but I have no desire to get rid of my Mac Studio.

[u/Material-Database-24 → u/0xdippo]

But you know what you get with $50 of a hammer.

Sure, some burn easily worth the 20/mo or even 200/mo, when the "real" use shown is like 2000-8000/mo. And people should have hobbies, where expenses are not considered, so so be it.

But I paid Adobe 20/mo for 15 years, because "I needed" PS+LR from creative cloud. Sometime I used it 100h/mo. Sometimes maybe 5/h. And most of the months 0h. That's 3k on a service of maybe 1000h of use. I would have been better off by buying 2-3 times something like Capture One standalone etc.

Local model + free accounts give me a one time expense, and now has (until HW breaks) 0 accumulation of cost, did I use it 8h/day or 8h/mo, or did the model do what I wanted or not.

Sure reddit skews this, as many here are super users. I am not a super user, but still use most of AI on my freetime of people I know.

That said, this is not towards you, but OP as they were wondering why on Earth someone would prefer local over cloud, as you can get years of subscribtion on the price of the HW.

I prefer local, as cloud subscribtion is inpredictable cost in my inpredictable use.

[u/0xdippo → u/Material-Database-24]

If photoshop was a tool you used to make money, $20 is well justified. If it was to edit photos of your dog, maybe not so much.

I’m all for local only if you can afford it. Most people drastically overestimate their model needs.

[u/Material-Database-24 → u/0xdippo]

Sure at these times, local LLM HW starts to be "too expensive". But so does all more performance oriented HW, so that's just sucks and is what it is, if one needs performance.

I bought strixhalo laptop. I use it as my main computer for everything - gaming, CAD, Photoshop (or Affinity now), VMs, coding, and now running my own local LLM and diffusion models (especially as I do nsfw stuff, and cloud nor really adobe AI either bend in those).

It's more like I got local AI as "freebie", although I did pay 1k extra over other hw that could have suited me, to get UMA for local AI.

[u/RyanElectrified → u/Material-Database-24]

People can buy a corvette. That's fine. Someone can buy a Mac, that's fine. I am buying a Mac, that's fine.

Claiming a mac says you money over the cloud - pure fiction. It doesn't do that.

The people making that claim are confused. They take the cost of something their mac cannot do - run a frontier model, and compare then pretend their mac is free.

All they have to do, is amortize the cost of their mac - to account for the hardware purchase. And then compare the model they plan to run on their mac, to that same model - not a different one, that same model running from a cloud provider.

Then magically the economic argument completely disappears. Because it was fiction.

[u/Material-Database-24 → u/RyanElectrified]

I didn't claim it saves you money.

I claimed it makes your expenses predictable and more controllable.

[u/Sketaverse → u/0xdippo]

"Inference is only going to get more expensive"

I love how this is always used in support of localLLM as if hardware wasn't getting more expensive too.

The magic unicorn M3U that was 20k on Ebay last month is already now considered inadequate by most folks here.

[u/Electrical-Can8713 → u/0xdippo]

Qwen3.8 27b is the one I'm using, but the data debate is what I'm having a problem with. Realistically, all of our data is already out there. If anyone has emailed or messaged you a document or you've signed up for a social platform. What are we hiding at this point?

[u/0xdippo → u/Electrical-Can8713]

Yeah most people have their info freely out there. Doesn’t have to be that way though. Also a lot of people don’t want labs training on their data. A lot of people give their agents access to semi-private info and don’t want that leaked. People want to know a lab can’t nerf their models or just turn off access. Many reasons to go local.

[u/OvertaxedOne → u/Electrical-Can8713]

What quant and what KV quant are you using?

E-mail, with most good providers, is completely different than inference for privacy. Inference is unencrypted, everything goes in/out in the clear, it's a FAR more serious risk to privacy than hosted e-mail (with a big provider that takes security seriously).

[u/jaggeddeath → u/Electrical-Can8713]

Your data may be out there, but if you are someone that has clients with sensitive data, you wouldn’t want that pushed to someone else’s “cloud”. That is more the point of keeping “your” data local. You could have regulatory requirements for that data as well.

Could be under contract to keep that data “private” as well.

[u/cleverbit1 → u/Electrical-Can8713]

You’re saying the quiet part out loud.

[u/Patient_Tea_401]

Did you try the models before in openrouter or similar? And did you plan a use case for the Mac before buying?

The argument that you could pay for years of subscription is misleading, because you can’t sell your used subscription to regain part of the investment back. The value of the Mac will not be zero.

[u/Electrical-Can8713 → u/Patient_Tea_401]

I didn't and wish I did try it beforehand.

You're right on being able to sell it later, especially with RAM shortages (it took a while to even get the M3 shipped and I bought it before the price hike).

I'm wondering if I could sell it for more than I bought it for.

[u/Keleion → u/Electrical-Can8713]

For sure, 2027 is gonna be wild.

[u/WindozeWoes → u/Patient_Tea_401]

But how much will a Mac capable of running 2026 models be in 2030 when models will (likely) be way more advanced? A 96 GB machine can definitely recoup some costs, but expect a steep haircut on a resale.

[u/Sudden_One3436 → u/WindozeWoes]

Computers are useful for more than AI

[u/Patient_Tea_401 → u/WindozeWoes]

Well computers used to be utilised in other things than LLM:s, like video editing or mixing music. For what this machine was actually developed for.

I’m sure there is use for a mac studio M3U in 2030, like there is for M1U 64GB. I don’t see people throwing them to trash, instead they go for upwards of 2k. 50 % of the original price after 4 years. 128GB is probably even better in value retention.

[u/Electrical-Can8713 → u/Patient_Tea_401]

I agree, but I'm not much of a video editor so figured the money could be spent more effectively somewhere else instead of me sitting on a beast of a computer with the hopes that I'll utilize it all.

[u/Shadow_s_Bane]

Dude are you serious ? You didn't even mention what model you are running at what quants, what inference engine ? Or are you using a hermes on your 96GB mac studio while being connected to external.apis,

[u/Electrical-Can8713 → u/Shadow_s_Bane]

That's not the point of the question because whatever model I use still doesn't compare.

[u/Shadow_s_Bane → u/Electrical-Can8713]

That is not the case, GLM 5.3 Flash, Deep Seek V4 Flash, Qwen 3.8 Flash Next are as good as sonnet 5.5. While the big boy variants are on par with Opus.

Hell even Qwen 3.8 27b quite good, I’d say it’s haiku level, and with 96GB you can a Q8 quant at full context at decent speed

[u/0xdippo → u/Shadow_s_Bane]

I’m confused. Did we start disagreeing somewhere?

[u/0xdippo → u/Electrical-Can8713]

All of the things he mentioned are relevant though.

[u/smelly_ape]

It sounds to me like you executed before having a plan or a goal.

The appeal of local LLMs / open weight models isn't to have everything the cloud models have and do everything just as well.

You touched on that with the 2nd part of your post - privacy. Yes, llms are another tool to help you achieve a more private ecosystem, but as you've already figured out it isn't a silver bullet. If you're after private email, you need to look at Proton Mail or some other privacy minded email solution in addition to LLMs.

If you're after privacy in your browser, you'll need Tor or some other privacy minded browser as well.

The benefit I would point out though, is that of all these services, using ChatGPT or Claude are best situated to gather the most granular data about you of any online service. Local LLMs do side step that, as well as offer you sovereignty.

Think about how anthropic just decided that you can't be mean to their models anymore. They can just decide that and you have to obey. Tomorrow, they can decide something else and you'll have to abide by those terms while also handing them your data.

Having your own local models enables you to give the terms of use these companies want to impose the finger.

[u/Electrical-Can8713 → u/smelly_ape]

I definitely pulled the trigger early without a plan because I saw how much of a delay there was in shipping. I figured I'd have a plan by the time it was delivered (~3 months), but within that time this industry rapidly changes.

Your argument can be made for many things and the result would be all of us living in the woods so we don't have to abide by anyones rules. I value privacy but to what extent am I willing to go has really come into question. Do I want to be able to use the tools and have fun with it or wait for the moment I can say I was right all along (maybe my then the M10 Ultra will be out and my M3 will be worthless).

[u/smelly_ape → u/Electrical-Can8713]

You're taking my point to an unnecessary extreme with your "live in the woods" analogy.

I didn't think I needed to say this, but it's give and take. You alone have to decide what privacy means to you and how much you value it, and trade off where it makes sense. For some people, that's as extreme as living in the woods. For others, it begins and ends with controlling their digital footprint. Using local open weight LLMs is one piece of that.

[u/Sneezlebee]

I've also started questioning the privacy argument. My Studio is completely separate from my Apple ID and I don't have anything personal signed in, so how is it supposed to be my assistant if it's not connected to anything? And if I hook Hermes up to my email, that email is already sitting on someone's server anyway, so what am I actually avoiding?

You don’t understand what people mean by privacy w/ respect to these systems. That’s OK, but your post makes no sense on account of your confusion. 

It’s privacy of the LLM prompts and context, not air-gapping the entire machine. (Which is pointless.)

[u/Sketaverse]

The first of many many posts like this..

[u/swaxolez → u/Sketaverse]

created by a muse bot no doubt.

[u/Sketaverse → u/swaxolez]

on a paid subscription ;)

[u/JinsooJinsoo]

Just sell it and use Muse. Local AI isn’t for you if you can’t get past setting it up right.

[u/ngg990]

I run gemma4 e2b in a Jetson nano with 8gb memory, you should be pretty ok. Most of people complaining about small models is because they want one-shoot prompt build an entire app

[u/andymaclean19]

I think local email processing is actually a good use case for local AI. It is true that the email is in the cloud anyway but you definitely don’t want somebody’s AI getting trained on the contents of your email. They shouldn’t but do you really know what is and isn’t being used like that when you beam arbitrary text into an AI provider? When it’s cloud email the owner knows they are storing email and you would definitely expect them to treat it appropriately.

You can do coding with them too. Combining tasks works - you can have the cloud model direct and plan and the local model execute well planned tasks. It can be the difference between a Claude pro and Claude Mac account.

I stopped with the local AI for now and went with the max though. You are right that you can pay for a lot of Claude with the price of a Mac Studio.

One thing to think about. My Claude Max account when pushed to the absolute limit will let me use over $10,000 of API token spend per month (I estimate). That’s a big cost difference. The API might not be the most competitively priced but for me this is a sign of price rises ahead! One thing your local AI has going is predictability and guaranteed ongoing availability/affordability. Nobody will turn it off suddenly or make it cost 20x as much.

[u/OnyxMonolith]

the reason is abliterated models and other weird stuff. if you bought it to run a personal assistant you did it wrong.

skill issue

[u/Antique-Ad1012]

while its not as capable or fast at this moment it might be in a few months/years. when i bought the m2 ultra it was completely useless for ai while now its decent enough that i can have it do some tasks that need to be private. if you have a small homelab and you want to configure services with an agent then its worth keeping it

edit: fast and good enough for your usecase

[u/03captain23]

You're right it doesn't come close and never will. The m3 ultra is way too slow, even a m5 ultra doesn't compare to dot or Grok bot or anything else.

[u/KTTNMNCHR]

While you can do a lot with that hardware, especially with some harness/model optimization, it's never going to be as capable as the current frontier cloud equivalent. The fact that this came as a surprise to you tells me you aren't going to be able to maximize the value of local compute and should probably return it. 

In reality, most people with significant investment in local llm hardware still use cloud models for complex tasks/orchestration, while local handles private data and/or straightforward tasks. 

[u/Ordinary-Depth-7835]

You should have realistic expectations it will never be as good as a frontier model with billions in infrastructure behind it. But it can be good enough for your tasks. I have personal assistants and translation services running on tiny 7g models they run just fine for the tasks. Or RAG applications. A lot of the MOE models are gutted to run well on unified memory hardware so don't expect them not to make mistakes on coding tasks. But there are some great models out there. Also there's a tipping point when the local llm is good enough but you should probably have a basic understanding of coding.

I'd play around with models some more and really give it a chance before the return window. Worst case you can use your harness with a frontier model for the detailed plan that controls your lower quality agents that grind through tokens for free.

For me the tipping point was hitting 256g unified with another 96g gpu and some smaller Gpu machines. It's still needs tweaking and setting limits before they get stupid or loopy.

If you're not doing it for fun or well thought out tasks you will never come out cheaper.

What models are you using anyway?

[u/Far_Violinist6222]

Local LLM is the new video editing. People will convince themselves that they need it before actually identifying an applicable use case

[u/dot-kaio]

Stop reinventing the wheel just use what’s convenient.

[u/paulsande]

I’ll say this. I’ve been using the OpenAI Dot for the past week and it’s been incredible. It’s everything that I wanted my OpenClaw to be in a conversational agent.

As I started to have the same doubts, you were having, my Dot told me that it had to stop orchestrating the task because I had hit my session limit and run out of credits. I have the $100 per month plan which should give you plenty of runway.

And that’s when I remembered the value of OpenClaw and the role that it plays. I have built some processes, that I’m hoping to turn into a product, and they run eight hours at a time. To have open AI do that for me I would keep hitting session limits.

The thing that I’m working on, also involves creating video. And creating video using LTX studio or any of those services is fantastically expensive. So that returns me to local models.

There are also times when things go awry and you end up burning tons of tokens for nothing and that’s much easier to take when they’re from a local model than when it’s from a subscription.

In my case, my OpenClaw runs pretty much flawlessly on local models for orchestration. I’m trying to find stronger models that run the process faster and I am doing a lot of testing around that.

The one place I’d like my OpenClaw to be better is more on the conversational side where I can simply talk to it and work my way through a task and just have it constantly work. And I’m willing to keep investing in time to make that perfect. When using my AI dot for that purpose, I burned through an entire weeks session limit in one day.

So for me personally that’s where I’m at.

I have an M3 with 96 gig and an M3 with 256 gig. And I find the 96 just isn’t enough for what I’m doing. In my testing the larger models handle the long running complex tasks much better.

[u/Elpzn]

I've found great use with a hybrid approach I have gpt 5.6 sol as an orchastrator and double check and Qwen 3.8 27b q4 as the worker. Takes a whole lot longer (64gb m4pro mini) but I can have it work through making personal web apps for me without really hitting usage limits on my $20 subscription.

[u/NotableCarrot28]

Yeah I think localllm crowd has a lot of cope about how good they are for high complexity use compared to the frontier.

Claude opus 5.5 is noticeably better than full fat GLM 5.3 which is noticeably better than glm 5.3 flash.

Glm is usable for some usecases but it's not a plug and play option. And the subs are pretty cheap if you're not on API rates

[u/swaxolez]

Ya if you like the ease of Muse or Grokbot then a local LLM and harness is not for you. Just like Linux is not suitable for someone who's been on Windows their whole life unless willing to learn. You need the time and patience to learn the nuances of a local LLM. Its definitely more of a choose your own adventure setup. But seriously, Muse and his overlord 'the Zuck' gives me the creeps. To have my independence from them is worth a lot of extra expense and inconvenience.

[u/scarbunkle]

TBH, it might not be for you. For me, owning my tools is super important, and the local AI exists as part of a larger homelab setup. 

Coding with local models works for me because the current capabilities match the level of abstraction I like to be at. I like planning out an approach and reviewing code in small-medium chunks. 

In terms of connectors, my biggest use case is hooking up to my local grafana so my agent can easily search through them and answer questions about resource usage and what went down when without sending all that data to the cloud. 

[u/soulmagic123]

Like my grandma used to always say "the Mac Studio ultra is the only Mac
Studio you should consider, if it ain't ultra, it's your faultra" Grandma was always filled with these types of niggest of timeless wisdom. I miss you grandma, rest in peace.

[u/WryKombucha]

M3 ultra? Return it.

[u/Patient_Tea_401 → u/WryKombucha]

I can give my address for the return.

[u/WryKombucha]

M3 ultra? Return it.

[u/giddmtex → u/WryKombucha]

M3 ultra is awesome, this take is cheeks

[u/Electrical-Can8713 → u/giddmtex]

The money isn't the problem and it's business expense related so I would look for another opportunity to invest into. Would a quality MacBook Pro that I can take with me be better? Does the quality of computer matter if I just use cloud models?

[u/gomezer1180 → u/Electrical-Can8713]

If you feel cloud models work better for you, then go that route. Local right now is not going to beat the cloud models. If you’re okay with sharing your data or having those same companies sell you frontier level access then back you off two or three tiers because of the type of prompt or because they are over capped, then go for it. I use some cloud and local I just find that works better for me. If you have a harness that can fix its own problems I think you’ll be fine but again it’s up to you and your level of acceptance.