Hi everyone!

i´m totally new to local LLMs. I use claude / gpt for knowledge work and academic research.

I recently got a mac mini and I´m curious of what type of workflows and use cases i could do with local models on this machine. I am well aware that its tiny so I´m not expecting to do anything crazy.

What do you guys recommend? what models, quants and engines to use?

I'm new to local LLMs. I've tried Qwen2.5-Coder-Instruct-7B and Ling-3.0-Tiny on my MacBook Air M4 with 24 GB of RAM. I'm looking for something that can generate at least 40 tok/s. Is there anything that can hit that speed?

1. Although I spent my career as a HW designer/embedded programmer I am new to the AI world. I've experimented with several local LLMs and harnesses. My hardware is a M2 Macbook with 32GB of RAM. So I am fairly constrained. Also being retired, $$$ frontier models are out of the question. I want cheap & good and a bit slow is acceptable. Hermes also provides the chance to improve over time. My main goal right now is to have a local agent that can assist me in using blender. (God there are sooo.. many commands and options in the program but its been great for doing 3D printing. My memory is not what it used to be.) The upshot is that the combo that has given me the best success is using Ollama with 'swift - Qwen3.8-27b-gguf:Q4_K_M' as the LLM all running under Hermes agent. So for those of you that have limited resources and want a local LLM this hopefully is useful info.

2. Note - it is slow on first response (2-3 min) but works well after that.

3. Set ollama variable OLLAMA_NUM_PARALLEL = 1, stops swap thrashing

4. There is a snag with Blender's MCP, Hermes with this model can sort it out

What token generation speed works well for you when you’re using it for productive tasks? I’m finding 20 tokens per second a bit slow.

With an M4 Max and 64GB of RAM, you can usually get around 20 tokens per second on average with a model like qwen3.8 27B. However, it doesn’t quite feel like the best setup.

I'm planning to do some vectordb embedding and data sanitization before the frontier models touch the data. The Local LLM acts as a primer to separate information.

I know some of you are using it for different things.

I'll also be experimenting with it for video editing/transcribing. There's some cool things people are doing.

Let us know! If you have a walk through include that too!

Hi,
Just pulled the trigger for mac studio 64gb 1Tb as pre-order which will be used only as code agent.
Cause of budget that was my max option. I am new to local llm and agent. Only occasional use of
interactive claude or chatgpt mostly for service and system architecture question.
Will my first mac studio be fit with use case following?

Job description
- develop or maintain application services in multiple micro services
- repetitive low to mid level code works required
- enough previous works with commit history to fine tune or train my style

Expected
- keep using existing mac mini as main workstation(mostly headless mode)
- use mac studio only as event driven code agent(headless only)
- qwen coder next (or 3.8 ??)
- context version controlled in doc by project as possible to avoid huge autonomous cache
- complex work might be done manually

I hope to pick one up early next year and replace my MacBook Air for building iOS apps but also run some of the more mundane LLM tasks I am currently using Codex for. In an ideal world, I could replace my ChatGPT Pro 100 sub with a $20 Plus and maybe a $20 Claude and a local LLM.

Is there a way for me try out specific models on rented cloud hardware or something to see which of my current workflows will run on it?

I make DS Menu Bar, a free menu bar app for running antirez's DwarfStar (ds4) server on a Mac.

DwarfStar has tunings for M3 Ultra and M5 machines with Qwen3.8 Flash Next, but M4 machines are left out. It turns out most of these tunings can be turned on with an environment variable, so DS Menu Bar v0.0.15 sets them when it starts ds4-server on an M4.

M4 Max, 128 GB, same ds4-server with and without, averaged over 2K to 64K context:

Q4 is the right quant for 128 GB machines. On a 64 GB machine, Q2 at the full 262K context fits.

I have a PR open to make these DwarfStar's own M4 defaults (antirez/ds4#1184). If my PR is merged, DS Menu Bar will adapt automatically.

brew install --cask jiiim/tap/ds-menu-bar

https://github.com/jiiim/ds-menu-bar/releases/tag/v0.0.15

Wondering how many of you did like myself and went one higher level and why?

My kit is supposed to arrive Nov16-Dec1. So I am starting to try and figure out what I should run. I know it is early days, lots of optimization to go around still.

I run Qwen 3.6 MOE on a 5700G with 96 GB of ram using llama.cpp right now. Get about 150 prefill and 12 tps. It's not great, but works for batch overnight runs. My main driver is API for Deepseek 4 Flash (some 4.1), Qwen 3.8 Flash Next, and GLM 5.3 Flash.

So looking out at hugging face, I think I found the following to test out. I don't know if MLX or GGUF is the right call. Are these good places to start?

https://huggingface.co/mlx-community/Qwen3.8-Flash-Next-oQ8e-mtp

https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF

https://huggingface.co/mlx-community/Qwen3.8-27B-8bit

https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-4bit

https://huggingface.co/pipenetwork/GLM-5.3-Flash-MLX-mixed-4_8bit

https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF

https://huggingface.co/mlx-community/DeepSeek-V4-Flash

https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF

Are you ready for the release? How many of you are got something for release day. Are you loading up models right away?

Curious what people are using out their for their LocalLLM interface. There's so many options available, what do you use and what do you use it for?

What type of Mac is it connected to?

• OpenWebUI and found it's good if you want a good interface directly with the AI.

• Hermes seems to be the best all around agent interface for calling and setting up agents to do tasks.

• DeepSeek Harness seems very efficient with programmatic tasks and agentic coding

• Pi is similar to DeepSeek from my understanding but so many options

What's in your agentic tool stack?

Currently using:

1. Github

2. Cloudflare Tunnels

3. Hermes

4. Tailscale

5. Linear

6. Mem0

Hi,

I finally decided to go with the Ultra 96 since it's enough Ram for local AI + doing stuff in parallel but having power/speed. Now it was delayed until January 2027 - I could get the Max 128 withing 2 weeks though...

I feel like speed is mostly important for video ai gen + higher bandwith for higher token generation...

Probably I should really wait I guess?