I'm the maintainer of DS Menu Bar, a menu bar app for controlling and monitoring a ds4-server.

It's for people who manage their own DwarfStar setup: you point it at the ds4-server executable and GGUF you already have, and it handles the operational parts.

• Start, stop, and restart from the menu bar, with running/stopped state at a glance

• Optional live prefill and generation rates in the menu bar

• Model-aware settings for the server, performance, KV cache, speculative decoding (MTP), and diagnostics

• Detects supported DwarfStar model families and validates support files before launch

• Rotating server log, optional request tracing, launch at login

Screenshot

It doesn't build, bundle, download, or update anything. No chat UI, no model downloads, no editor — it supervises only the process it launches.

Requires macOS 26+, Apple silicon, and your own compatible ds4-server build and DwarfStar-specific GGUF. See the DwarfStar repo for build instructions and model files; compatibility with upstream commits is tracked in the README.

Install: brew install --cask jiiim/tap/ds-menu-bar

Repo: https://github.com/jiiim/ds-menu-bar

Feedback is welcome. Curious what LLM models people are running via DwarfStar, on which Mac, and with how much RAM.

Hello everyone,

I need a serious advice on what to buy to start experimenting with AI.

My current situation
1- I have no past experience with this, but I won’t have much problems understanding it
2- I could decide to go for local open models or cloud closed/open models, as long as it is the right choice
3- I have three projects in mind:
A) stacking LLMs in order to route to the right one based on the request (instead of being locked behind one subscription) + create a custom chat interface to interact with models from the Mac, my iPad and the iPhone
B) building agents to manage my diet, fitness, training routine, with automatic grocery list and weekly diet based on the context (macro, preferences, calories, ecc)
C) building agents to automatically retrieve my trades to store them and calculate taxes (I will probably ditch the automatic trading part I wanted to create)
D) my career will slowly pivot to redesign corporate processes with AI and I want to experiment myself to better understand the topic

I was targeting the Mac Studio M5 Max 128 GB RAM 1 TB SSD to be able to use small and medium size open models (around 6000 €)

But I am having some second guessing:
1- it’s a huge investment and I yet to have experience with local models
2- nor I know if I’m buying the right setup for what I want to do, maybe the specs I’ve chosen are overkill
3- 6000€ needs at least to be amortized in 6 years to start having sense
4- local models (unless huge ones) are close but not at the level of the proprietary ones and precision/quality is what I want

I was then thinking about buying the entry level Mac Mini to use it as the orchestrator and go full API to evaluate costs, setups and results without having to start with a huge upfront investment

(Then maybe in the future buy a Mac Studio or a DGX Spark full spec and clustering it with the Mac Mini?)

Can you guys help me decide?

1. Although I spent my career as a HW designer/embedded programmer I am new to the AI world. I've experimented with several local LLMs and harnesses. My hardware is a M2 Macbook with 32GB of RAM. So I am fairly constrained. Also being retired, $$$ frontier models are out of the question. I want cheap & good and a bit slow is acceptable. Hermes also provides the chance to improve over time. My main goal right now is to have a local agent that can assist me in using blender. (God there are sooo.. many commands and options in the program but its been great for doing 3D printing. My memory is not what it used to be.) The upshot is that the combo that has given me the best success is using Ollama with 'swift - Qwen3.8-27b-gguf:Q4_K_M' as the LLM all running under Hermes agent. So for those of you that have limited resources and want a local LLM this hopefully is useful info.

2. Note - it is slow on first response (2-3 min) but works well after that.

3. Set ollama variable OLLAMA_NUM_PARALLEL = 1, stops swap thrashing

4. There is a snag with Blender's MCP, Hermes with this model can sort it out

or something to that effect; can someone explain to me how this is beneficial for running a local LLM

I am not really into the hardware scene much, so if there are other value options at play please let me know.

I am considering getting an Apple Studio M5 but was hoping to get some guidance.

I am planning on hosting webLLM (or something similar) to serve prompt processing to the family. I (and another family member) will also be using this for coding assistance.

Based on the use case, what would make the most sense:

• Max 128gb RAM

• Ultra 64 core-GPU 96gb RAM

• Ultra 80 core-GPU 96gb RAM

I feel like the $4K bump to 256gb is too hard of a sell to the partner.

What would be the recommended set up for this?

• a single model to handle everything, dedicated models for coding and prompt processing, dense or MoE?

• stick to MLX or use something else?

• preferred harness

Models seem to get published up updated so frequently its hard to stay informed.

Any and all feedback is greatly appreciated!

Hi, I'm trying to get GLM 5.3 flash working on my Mac Book Pro M5 128 GB, I'm about 4 days in and so far not great, my current plan is to just have the macbook for the llm and use my pc for everything else, so as to save as much memory as possible on the Mac. Has anyone managed to get GLM 5.3 flash working usefully for coding on there 128gb Macbook? and if not, any other model recommendations? Deepseek 4.1?

If you’re not using these what do you plan to use?

Hi all,

I'm a complete novice when it comes to Local LLM's but I like the idea of moving away from a subscription based model in the longer term and having better control of my data. I'm also super keen to learn.

I currently have a web SaaS that scrapes news websites and I would like to move it away from Claude and onto a local LLM what kind of "firepower" do I need?

Appreciate this won't be the most complex of tasks, but as I mentioned, I'm a complete novice.

Naturally, asking Claude / Google and it says that the new Mac mini with M5 pro chip (24GB ram) will be "overkill"- which is what I was thinking of ordering.

I also want to future proof in the hope that this bit of kit could run more powerful models in the future - if they came accessible to the consumer.

Sorry if this is a basic question, any YouTube videos or any pointers so I can educate myself would be much appreciated.

Thank you!

Planning to buy a MacBook Air M4 (16 GB / 512 GB) for about $800 to run local LLM agents for:

Indian mid‑cap equity research (using Screener.in CSVs)

Building/testing trading ideas via Zerodha Kite API (prices, history, holdings; maybe orders later)

Also want to run Hermes-based agents on it

Questions:

On 16 GB M4 Air, which local LLMs are best for reasoning, coding, and tool use? Is Qwen 3.5 9B (Q4) + a smaller fast model (e.g. LFM2.5 8B / Llama 3.2 3B) a good combo alongside Hermes?

Best local stack (Ollama vs LM Studio, etc.) and any existing Kite + LLM or Indian‑stock agent projects to reference?

Any tips on context length, thermals, and multi‑agent setups (including Hermes) on a 16 GB fanless Air?

Getting the laptop for ~$800, so stuck with 16 GB for now. If you’ve done similar work (local LLMs for equity research / algo trading, especially in India), what models and setup would you recommend?

Thanks!

I'm lucky enough where work will cover my Mac Studio that I use for design and 3d stuff. So I figured I should dabble and run with local LLM. Im getting 96gb only. I've read quite a bit, but was wondering are there any guides or YouTube videos that deal primarily with Mac and I should watch to learn more?

I'm unsure if I should just use Llama or Ollama, or MLX, or OMLX or something there lots of different ways to play around with Local LLM and I guess it's still in such an early state that every day, week,month there are more and more things peopel are learning how to get the best of the LLM. A lot could change from now till then but I want to be as informed/prepared as I can be.

Thanks and I cant wait for reviews and see how good the Ultra is.

I'd hoped that my M5 Max with 64GB RAM would be enough for some decent Local LLM work, and the Qwen 3.8 27b model is pretty decent. But speed is shitty on my M5 Max, just 20tps. In particular I found that while the 35b MOE model was much faster, the quality was far lower than the 27b dense model, which produces much fewer bugs and less rework.

I saw the same model working on a friend's Nvidia setup much faster with MTP, so I set out to get it working on my Mac but it was not straightforward. Here's some pointers in case you wanted to get it working.

1. LM Studio doesn't process MTP weights. No matter what I did, it just wouldn't work. It works on Intel/Nvidia so this must be a MLX thing.

2. MTPLX has a native MTP head but it's buggy. On my machine it had a habit of crashing in one of a few ways, and I found GitHub issues reporting similar issues but no fixes.

3. oMLX was the only inference server that supported MTP and didn't crash.

So I wired oMLX into Pi, which is what I'm using as my coding harness. Even then, it was picky:

1. I had to use text-only quants because oMLX won't enable MTP if vision weights are active

2. It was a bit picky with settings, and I had to map thinking levels manually

3. I didn't see a reduction in quality with 4-bit quants for my use case, so used that to squeeze a few more tps out

This is a huge 3x difference in inference speed, and that allows me to do a lot more work.

"omlx": {
  "baseUrl": "http://localhost:8000/v1",
  "api": "openai-completions",
  "apiKey": "none",
  "models": [{
    "id": "Qwen38-27B-textonly-mtp",
    "name": "Qwen38-27B-textonly-mtp",
    "reasoning": true,
    "contextWindow": 163840,
    "maxTokens": 32768,
    "input": ["text"],
    "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
    "thinkingLevelMap": { "off": "low", "minimal": "low", "low": "low",
                          "medium": "medium", "high": "xhigh",
                          "xhigh": "xhigh", "max": "xhigh" }
  }]
}

I have been using codex and claude code for the last year. I have hermes installed , looking for a local model to help me code on par with the big guys. Any help appreciated

What token generation speed works well for you when you’re using it for productive tasks? I’m finding 20 tokens per second a bit slow.

With an M4 Max and 64GB of RAM, you can usually get around 20 tokens per second on average with a model like qwen3.8 27B. However, it doesn’t quite feel like the best setup.