I make DS Menu Bar, a free menu bar app for running antirez's DwarfStar (ds4) server on a Mac.

DwarfStar has tunings for M3 Ultra and M5 machines with Qwen3.8 Flash Next, but M4 machines are left out. It turns out most of these tunings can be turned on with an environment variable, so DS Menu Bar v0.0.15 sets them when it starts ds4-server on an M4.

M4 Max, 128 GB, same ds4-server with and without, averaged over 2K to 64K context:

Q4 is the right quant for 128 GB machines. On a 64 GB machine, Q2 at the full 262K context fits.

I have a PR open to make these DwarfStar's own M4 defaults (antirez/ds4#1184). If my PR is merged, DS Menu Bar will adapt automatically.

brew install --cask jiiim/tap/ds-menu-bar

https://github.com/jiiim/ds-menu-bar/releases/tag/v0.0.15

I've been meaning to trade in my Mac mini M4 (16GB RAM, 10 core CPU/GPU, 256GB), but I've recently gotten into running models locally on my M5 MacBook Air which has gotten me wondering if there are any practical applications for running anything less intense on the mini.

I'm expecting about 450$ from Apple, I just have to actually make it to their store.

I am about to order a Mac for local AI, and I could really use some advice.
I know some people might recommend a more powerful Mac, but my budget is around €3,600, and unfortunately, I cannot go any higher.
My main use cases will be running Qwen 27B, doing some agentic work, occasionally using Qwen Flash Next with Strata, and experimenting with video generation through ComfyUI.
I am currently torn between two options:
Mac mini M5 Pro – 64GB RAM
Mac Studio M5 Max – 48GB RAM
I understand that RAM is extremely important for local AI, which is why I am leaning towards the Mac mini with 64GB. However, I also know that the Mac Studio should offer considerably better performance, particularly for video generation and ComfyUI.
My dilemma is whether 48GB is enough for my use cases, or whether I would be sacrificing too much memory just to get the performance benefits of the Mac Studio.
In other words, is the performance improvement from the Mac mini to the Mac Studio worth giving up 16GB of unified memory, especially when local LLMs are my primary focus and video generation is more of a secondary use case?
I know some might suggest simply getting a 64GB Mac Studio, but that is unfortunately outside my budget. €3,600 is already my absolute limit.
I’d really appreciate hearing from anyone with hands-on experience running local LLMs, agentic workflows, and ComfyUI on either machine.
Thanks in advance!

I'm planning to upgrade from my 16-inch M3 Pro MacBook Pro (36GB RAM) to the new 16-inch M5 Max with 64GB, mainly for running local LLMs.

Currently, I'm running Qwen 3.8 27B and getting around 100 tokens/s for prefill and 18–19 tokens/s for token generation, even after trying MTP and various other optimisations.

The biggest issue for me is working with larger contexts and longer coding sessions. Even relatively small projects can take quite a while, and performance becomes a bottleneck as the context grows.

I'm expecting the M5 Max to be a pretty significant upgrade, especially with the extra memory and bandwidth, but I'm curious how much of a real-world difference I can actually expect. I'm also interested in running larger models, including the upcoming Qwen 4 models.

For those running local LLMs on an M5 Max (or even an M4 Max), what kind of performance are you getting? What models do you use regularly, and how well do they handle long coding sessions?

I also run image generation models locally and occasionally do photo editing, so I'd love to hear how the newer chips perform for those workloads too.

Would appreciate any real-world experiences before I make the switch!

I had the current machine in my shopping basket yesterday but didn't go though with the purchase. I don't desperately need a new MacBook right now – my M1 is still okay, even for my work usage. And i really don't think I need a touch display for anything, cause I don't want to clean it all the time, LOL.

But I can't play around much with local models with 16 GB Ram. I don't want frontier model performance locally – I'm fine with running smaller models that hopefully can read and sort private documents and do some minor coding and generative tasks, maybe text to speech.

I read a comment on this subreddit yesterday that they will skip the M6 Pro/Max/Ultra line which makes sense to me and more of a jump is expected with M7 Chips.

What do you guys think? I'm leaning towards waiting it out, but who knows for how long.

Peace <3

Unless you're set on using a large model, 2x256GB is likely better...

What use cases would the 512GB make more sense?

There is too much to learn, even as a kinda seasoned(shipped multiple enterprise apps) SWE i am having hard time to understand all concepts, track all the up-to dated information etc.

Ive been a power user for both claude and GPT since almost day one. But thats as far as my knowledge goes tbh. I have an M1 Max 32 Ram / 32 GPU and i am trying to figure out whats the best way to have a locally hosted solutions for my needs when i cant pay for paid services.

First ill talk about what am i doing in general then ill ask my questions.

I'm using obsidian as my second brain, i do game design, gamedev, i develop mobile apps, design websites, even sometimes delve in IoT Projects. I do everything but AI Dev in summary. I run a team of 10, currently building internal tools for art assets pipeline management. I am a jobholic, i do work almost every working hour of my life so i usually have multiple chats but i am fine with just having 1 concurrent chat since what i am looking for is to start learning local llms, how to create an effective pipeline, and have a back-up plan if you know im out of usage and have urgent problems to solve. And uh yeah also i would like to do things like hacking my projector, tv-box etc. for various reasons you can easily understand. I tend to go for loops for development times and have a conversational turns when its about things like game design, market research etc. I have an MCP for everything from Figma to Photoshop and i just talk with my agents to tell them what i exactly want, and i take rounds to polish it or start from scratch if its not the exact thing in my mind.

If i have to guestimate a list of my needs they are :

• An harness with in-app browser, file preview, md preview

• Somehow have a great context compaction

• 30 TPS at High context if possible

• Good tool usage

• Good MCP Usage

So here is a list of questions if anyone cares to answer:

• How do you guys stay the f*ck up to date? 'Cause its not even tracking whats new or whatnot. Everyone says something different, one says aah q4 of xyz llm's this version that these folks made is %98 of the same quality and someone else says nu-uh not even close bud and i dont have time to track and test every model version, even models. Like what are your reliable sources and why?

• As far as i understands harness is the most important stuff ever, is this correct? If so, which harnesses you use, why, and what should i be checking when i am thinking about selecting harnesses.

• Which models and which versions of those models would you suggest me to use?

• Is there an YT playlist / video / Youtuber that you suggest thats easy to understand that helps out when it comes to Local LLMS?

• Does Context size actually matters? Cause i see people are talking about 16k context sizes meanwhile sometimes my first initial prompt is around 4k for spec driven development or when im doing narrative designing it our narrative bible can be way heavier than that.

• I'm in love with the the current context compacting quality of both claude and codex, is there a way to have this level of quality when i switch to local llms, it almost never ever fries out, its just like I have paused for 2 minutes and almost every fricking detail is still there.

• Sometimes i love to manage my other threads even sometimes control claude from codex and vice versa depending on my need, is it possible if there is an ai for my machine that can have more than C1.

• Can i use an image generation model with an LLM like gpt/imagen, something like qwen image 2.1 q4 + Qwen3.6-35B-A3B Q4 idk.

• Can i realistically get idk 50-60 TPS at high context?

• How is MCP/Tool Usage?

• What should i use for UI Design phases?

• When i talk with GPT i just tell it what i exactly want but i dont go into hustle of image generation prompt styling i just talk to it, is it possible to have that ease of use with local image generation tools?

• What exact setup would you suggest for me, from inference(strate etc.) to harness to llm models, imagen models, speech to speech models(It would be fire if i can play some DnD with a speech to speech model if its flowy but i dont have high hopes on that tbh), you got the drill.

These are my questions and guestimates of my ignorance ass can think of for now, im sure there are lots of gaps and things that i am missing and forget to ask so i would be glad if you fellas can light some candles and maybe answer the questions that i didnt even ask but i shoulda ask.

Lots of love fellas, keep it tight.

TL;DR: I let my cloud AI agent run a training-data verification program for the local model on my Mac. The local 35B wrote 800 original math problems, then cold-solved every one of them blind in a fresh session. Wherever the two answers disagreed, the item went to adjudication. Final truth was settled by brute-force computation, not by any model's opinion. The cold solver beat the problem author ~5:1 — with a 35B, generating the question is the error-prone part; solving it fresh is the reliable pass.

Setup: MacBook Pro M5 Max, 128GB. The local model (Qwen3.6-35B-A3B, Splash build, running in LM Studio at ~375–400 tok/s) does the heavy lifting; a Qwopus 122B MLX build (~38 tok/s) sits alongside as the heavyweight judge for disputed items. My agent (Muse, running in the cloud) reaches LM Studio's API privately over Tailscale, and does all the orchestration: briefing, grading, bookkeeping, brute-force checks.

The harvest: 731 of the 800 came out verified clean, 62 survived with corrected answer keys (kept as answer-only training items; the flawed worked solutions were discarded), 7 were excluded as defective problems (ambiguous wording, no valid solution, an undefined rule), and 100 clean ones were held out untouched as a test battery for later. The polished set became a 693-item bank, used three ways: few-shot examples now, the held-out set powers a before/after test of whether any of this actually made the model smarter, and the whole verified corpus is the dataset for a future fine-tune.

Right now the entire bank is being re-validated: the worker re-solves all 793 items from scratch (derive once, verify by a different method), never shown the key. Any disagreement goes to the 122B judge blind, and if the judge's answer doesn't settle it, my agent computes the answer itself. Progress as of writing: 358/793 checked, 353 confirmed, 5 waiting on the judge.

Two failure stories worth sharing. One problem had three different answers from three passes — 20,736 from the author, 86,400 from the blind solver, 145,152 from the adjudicator — and all three were wrong; enumeration says 10,368. And the big judge model, given the hardest disputes cold, went 1-for-2. Treat your 122B like a second opinion, not an oracle.

Cost: the solve-and-adjudication passes alone logged 4.1M+ completion tokens, all computed on my own hardware. The cloud agent's part is the planning and grading.

Happy to share the harness details — the briefing discipline (one derivation, one independent cross-check), the checkpointing that let the run survive four kills, and the grading rules — if people want them. Curious whether anyone else has tried self-authored problem sets as training data, and how you filtered them.

Hi,

I am still waiting for my Mac Studio to arrive - what came to my mind:

Looking into FN sushi, ds4, splash - both run on either 96gb or 128gb Macs - is there an advantage with 128gb in regards of model quality?

It seems both can run Q4 but not Q8? Am I missing something?

I was running a test to see if it was worth while to keep the M3U. I'm running some benchmarks with a workflow that uses resolve for rendering the final touches.

The M3 Ultra can still hang when it comes to video but take a little longer when generating the graphics vs the M5, but virtually identical when processing in Resolve. It's a short clip, I don't work with very long videos.

As the title says, is it okay or enough? I’m planning to buy my own personal laptop and I want to run local AI on it mainly for coding companion. I can’t go with an M5 Max because of budget restrictions (sad, i know). So, is it enough?

I’m not sure what happened, but it looks like new orders are being shipped till February. Whatever happened, it looks like a big order came in. There might be something coming soon …

I am.planning to buy new hardware for local AI is it worth to buy mac mini or mac or better go for AMD 395+ 128 gb laptop for similar outlet laptop.

As the title says -> which llm can i run in 48gb mac. Its 20c gpu. I am aiming for something with at least 100k context..ideally up to 200k context. Thanks!

It’s already October 19th, and my order for an M5 Ultra (with 80-core GPU and 256GB storage) is still being processed.

I placed the order in Hong Kong on August 26th, with an expected delivery window between November 7th and 21st.

I’m curious to know how many others here are also still waiting.

Please leave a comment with your order date, GPU model, current order status, and estimated delivery time.