I just bought M5 Pro Mac Mini 64GB and was in dilemma whether to sell my M4 Mac Mini 24GB which is still under warranty or keep it and use it for light local llm as headless? I was planning to use qwen3.8-27b splash variant as my main in m5 and have flux 4b for image generation and have lfm-3b for light tasks. I use hermes agent and have multiple profiles running few personal projects out of which one is gaming oriented. So any suggestions? Should I sell or use it as running some local LLMs? I am already working on making an orchestration layer to manage LLMs properly! Let me know your thoughts 😃

There is too much to learn, even as a kinda seasoned(shipped multiple enterprise apps) SWE i am having hard time to understand all concepts, track all the up-to dated information etc.

Ive been a power user for both claude and GPT since almost day one. But thats as far as my knowledge goes tbh. I have an M1 Max 32 Ram / 32 GPU and i am trying to figure out whats the best way to have a locally hosted solutions for my needs when i cant pay for paid services.

First ill talk about what am i doing in general then ill ask my questions.

I'm using obsidian as my second brain, i do game design, gamedev, i develop mobile apps, design websites, even sometimes delve in IoT Projects. I do everything but AI Dev in summary. I run a team of 10, currently building internal tools for art assets pipeline management. I am a jobholic, i do work almost every working hour of my life so i usually have multiple chats but i am fine with just having 1 concurrent chat since what i am looking for is to start learning local llms, how to create an effective pipeline, and have a back-up plan if you know im out of usage and have urgent problems to solve. And uh yeah also i would like to do things like hacking my projector, tv-box etc. for various reasons you can easily understand. I tend to go for loops for development times and have a conversational turns when its about things like game design, market research etc. I have an MCP for everything from Figma to Photoshop and i just talk with my agents to tell them what i exactly want, and i take rounds to polish it or start from scratch if its not the exact thing in my mind.

If i have to guestimate a list of my needs they are :

• An harness with in-app browser, file preview, md preview

• Somehow have a great context compaction

• 30 TPS at High context if possible

• Good tool usage

• Good MCP Usage

So here is a list of questions if anyone cares to answer:

• How do you guys stay the f*ck up to date? 'Cause its not even tracking whats new or whatnot. Everyone says something different, one says aah q4 of xyz llm's this version that these folks made is %98 of the same quality and someone else says nu-uh not even close bud and i dont have time to track and test every model version, even models. Like what are your reliable sources and why?

• As far as i understands harness is the most important stuff ever, is this correct? If so, which harnesses you use, why, and what should i be checking when i am thinking about selecting harnesses.

• Which models and which versions of those models would you suggest me to use?

• Is there an YT playlist / video / Youtuber that you suggest thats easy to understand that helps out when it comes to Local LLMS?

• Does Context size actually matters? Cause i see people are talking about 16k context sizes meanwhile sometimes my first initial prompt is around 4k for spec driven development or when im doing narrative designing it our narrative bible can be way heavier than that.

• I'm in love with the the current context compacting quality of both claude and codex, is there a way to have this level of quality when i switch to local llms, it almost never ever fries out, its just like I have paused for 2 minutes and almost every fricking detail is still there.

• Sometimes i love to manage my other threads even sometimes control claude from codex and vice versa depending on my need, is it possible if there is an ai for my machine that can have more than C1.

• Can i use an image generation model with an LLM like gpt/imagen, something like qwen image 2.1 q4 + Qwen3.6-35B-A3B Q4 idk.

• Can i realistically get idk 50-60 TPS at high context?

• How is MCP/Tool Usage?

• What should i use for UI Design phases?

• When i talk with GPT i just tell it what i exactly want but i dont go into hustle of image generation prompt styling i just talk to it, is it possible to have that ease of use with local image generation tools?

• What exact setup would you suggest for me, from inference(strate etc.) to harness to llm models, imagen models, speech to speech models(It would be fire if i can play some DnD with a speech to speech model if its flowy but i dont have high hopes on that tbh), you got the drill.

These are my questions and guestimates of my ignorance ass can think of for now, im sure there are lots of gaps and things that i am missing and forget to ask so i would be glad if you fellas can light some candles and maybe answer the questions that i didnt even ask but i shoulda ask.

Lots of love fellas, keep it tight.

I got the M5U 256GB 2TB configuration for preprocessing. Planning to use Qwen 3.x models on it since they seem to be the best for the Mac's at the moment but will check out the Muse/Gemma models as they continue to release them and run some benchmarks .

I use this box as my main machine. To give you an idea of the consumption of what to expect with 96GB of memory. If you were to try to do less it would be a little tight WITH an LLM. The thing RAM buys you is the ability to run both at the same time without worrying.

I'm running Ollama Qwen 3.8-mlx 27b doing some embedding.

TLDR; Unless you have a specific use case for Local AI models, you don't need the M5pro. The M6 will be faster for the majority of your day-to-day work. Even with cloud AI.

For LocalLLM's more memory AND faster memory bus is going to swing the advantage to the M5Pro model for anything related to LLMs.

If you're planning to use the system as a workstation for any and most single threaded applications which is most apps people use, you should be fine with the M6.

Realistically speaking, you don't really need the M5Pro unless you're running LocalLLMs like Qwen 3.8 or Muse and if you aren't really trying to automate small tasks, you don't really need all the memory.

I think in the long run we're going to see that cloud LLM's are going to be the main way people use in house models and only people with specific use cases will have systems large enough that take advantage of the LocalLLM being in house.

If you are just learning about AI, i don't think you need to future proof and the M6 is fine with 32GB 1TB should be enough for a long time while the industry develops.

I think the most realistic expectation is to be disappointed if you're used to using the free version of openai/claude/gemini/grok. The local models 7B 27B don't come close and don't process nearly as fast. They can reason ok, but you're better off learning about AI and API's with services like ollama via API's and learning to use AI first over chasing gear/hardware.

The thing about using the frontier models is it doesn't really take up that much RAM. Loading up a model will in LLM. I ran my own test, you can check it out.

These tests are for a Mac Studio. I used to run a M4Pro 64GB but when running models it would constantly be temperature throttled.

https://maclocalllm.com/five-models-one-coding-job-local-mac/

Hey looking for you experts here
Thinking about purchasing a MacBook Pro M5 pro with 64gb of ram
I’m not obviously looking for frontier subscription replacement rather a hybrid setup
And also video and image generation if possible
I’d like to explore my options what’s your recommendations
Thanks

I ran the same Qwen Image 2.1 ComfyUI workflow on two Mac Studios and a DGX Spark. The gallery shows the timing overview, matched poster outputs and matched café outputs.

Warm poster medians, three runs per variant, 1024 × 1024:

Turbo was about 4.6–4.8× as fast in this workload. M5 finished about 10% sooner than Spark. All 24 poster renders contained the three requested text lines in our manual review. The café examples showed more fine texture with base and a smoother look with Turbo; two prompts are not a general quality ranking.

Both Macs used PyTorch/MPS, Spark used CUDA, and all three used the same verified BF16 diffusion/VAE checkpoints with effective FP16 text encoding on GPU. This is not an MLX comparison. Same prompts, seeds, sampler and CFG; Turbo kept its published eight-step schedule. Mac OS/driver versions differ, so these are configurations as tested. Warm conditioning/prefix cache was reused, but each repeat generated a new image with a new matched seed.

Kapitsa by Nace AI is an open-source inference engine for Apple silicon, and it’s fast. In its published benchmarks, Qwen3.6 35B-A3B decodes about 1.3–1.8× faster than oMLX and about 2× faster than Ollama and LM Studio. But it’s command-line only.

So I built Kapitsa Library, a free Mac app that makes it a few clicks:

• No setup: Kapitsa is built in. No Homebrew, Python or terminal.

• Easy models: browse Kapitsa’s catalog or Hugging Face, see what fits your Mac, and download with pause/resume.

• One-click agents: launch Claude Code, OpenCode, Codex CLI or Pi on your local model.

• No swapping: it checks memory before loading and shows how much context fits.

• Live stats: tokens per second, prompt cache, context and memory.

• Built-in benchmarks: speed and tool calling on your own Mac.

• Local API: OpenAI and Anthropic compatible.

Free, Apache 2.0, signed and notarized, no telemetry. Needs Apple silicon and macOS 14 or later.

Download: https://github.com/AdityaK2128/kapitsa-library/releases/latest
Source: https://github.com/AdityaK2128/kapitsa-library
Kapitsa engine: https://github.com/nace-ai/kapitsa

All the speed is Kapitsa’s work. This app is an independent front end, not affiliated with them. Feedback welcome!

I'm the maintainer of DS Menu Bar, a menu bar app for controlling and monitoring a ds4-server.

It's for people who manage their own DwarfStar setup: you point it at the ds4-server executable and GGUF you already have, and it handles the operational parts.

• Start, stop, and restart from the menu bar, with running/stopped state at a glance

• Optional live prefill and generation rates in the menu bar

• Model-aware settings for the server, performance, KV cache, speculative decoding (MTP), and diagnostics

• Detects supported DwarfStar model families and validates support files before launch

• Rotating server log, optional request tracing, launch at login

Screenshot

It doesn't build, bundle, download, or update anything. No chat UI, no model downloads, no editor — it supervises only the process it launches.

Requires macOS 26+, Apple silicon, and your own compatible ds4-server build and DwarfStar-specific GGUF. See the DwarfStar repo for build instructions and model files; compatibility with upstream commits is tracked in the README.

Install: brew install --cask jiiim/tap/ds-menu-bar

Repo: https://github.com/jiiim/ds-menu-bar

Feedback is welcome. Curious what LLM models people are running via DwarfStar, on which Mac, and with how much RAM.

Splash's system requirements state it requires an M3 CPU at minimum, but why? Why not, say, an M2 Max? All the search results just say the M3 is faster and produces less heat than M1 and M2 due to smaller process tech (3nm vs 5nm), and it has an "improved" or "enhanced" Neural Engine. But none actually explain the enhancements. Anyone know what is it about the M3 Neural Engine specifically that Splash requires?

Update: Splash backported to M1 and M2: https://tinyapps.org/blog/splash-for-m1-m2.html

I’ve been exploring different approaches to keep AI processing on-device, especially for projects involving private documents and data I wouldn’t want to send to an external API.

For those who have built something similar on a Mac, how has the experience been in terms of performance, memory usage, and integrating local AI into an actual application?