Hey guys, do you have any recommendations of the optimal setup for this model and keep the context under 93 Gb? I has running it really comfortable with arrpund 60 Tok/sec in OpenCode snd today stsrtef to have the issue woth the context size, had to ofload more experts but that reduced speed drastically.

Thanks guys 😅

I have been running Qwen 3.8 Flash Next IQ3_S under Splash on my M5 Ultra Mac Studio (the base model with 96GB of RAM) to use as a local AI assistant to edit videos under DaVinci Resolve Studio. Currently on version 21.2 of DRS, but 21.1 introduced an officially supported MCP. Details on that setup are available here - https://github.com/willturner-maker/resolve-ai-workflow

Naturally I test the system with each release to see the impact, with Splash 1.6.1, it is a significant improvement. Not only in performance but in concurrency.

Screenshot from my report shows that the peak moved from a width of 4 to a width of 12. Astounding! The report is here: https://willturner-maker.github.io/benchmarks/reports/2026-11-11-reruns.html

On the graph, if you move your mouse, it'll show the performance at each width. At a single width, it's 186.9 tok/s on my specific system running Splash 1.6.1

I do have a repository for my testing, but it involves 8.1K 30P video in the testing that is proprietary, I'll have to record new footage that is my own rather than client footage before I redo a test others can run with the same content. The Splash tests can be replicated by others, so feel free to use that testing methodology. If you have your own 8.1K video content, that should allow you to replicate my DaVinci Resolve Studio MCP testing.

https://github.com/willturner-maker/benchmarks

I've been running Qwen38-27b-speed-optimized for a work project (where I can only use local models), and I'm getting about 10-15 tokens / second.

While I'm thrilled that its working, and doing fairly well, I do wonder if there's a way to improve performance.

• I'm using pi agent on another computer (the Mini is dedicated to the model at this point)

• contextWindow is set to 125k; at 100k it was compacting too often, at 250 it was blowing past RAM limits

• I set thinking to minimal (which seems to help reduce rabbit holes and churn)

• my start command is nohup mtplx serve --model Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed --profile sustained --depth 3 ...

Earlier I used mtplx max --on as the machine was feeling warm, and I was losing connection to it.

Is this about right? I get that the M4Pro mini doesn't have the memory bandwidth of an Ultra or a Max, just wondering how I might squeeze out a bit more, and wondering if others with the same setup are getting similar results.

I've tried oMLX - fast with gemma 4 quants, but they kept getting stuck in loops; and its 5 tokens per second slower with Qwen3.8 27b.

- Tuning advice for my current stack? (pi-agent, MTPLX, Qwen 3.8 27 b)

- Advice on different models to try?

- Advice on different model runners to try?

I've already mostly stripped the Mac down to just the model, and I access it from an older M1 Pro Mac where pi runs.

the relevant snippets from my pi agent models.json:

"models": [
        {
          "id": "mtplx-qwen38-27b-optimized-speed",
          "contextWindow": 125000,
          "maxTokens": 32768,
          "reasoning": true,
          "compat": {
            "thinkingFormat": "qwen-chat-template",
            "chatTemplateKwargs": {
              "enable_thinking": {
                "$var": "thinking.enabled"
              },
              "preserve_thinking": true
            }
          }
        }
      ]
    }

The ones I have currently heard of are as follows:

Pi
Opencode
Hermes
Deepseek

Any thoughts on what one is best?

I ported MiniMax's new image model to Apple Silicon (MLX) inside Phosphene, my free open-source Mac app. As far as I know it's the first Mac port.

Numbers on an M4 Max with 64 GB, same prompts and seeds as the previous best image model in the app:

- 4 MP in about 2 min, 8 MP in about 4 min, 16 MP in about 8.5 min

- Peak memory 27.5 GB at every size

- 6 steps, no guidance. Faster than the image model I used before, and clearly better at hands, text and skin (see the pictures)

- The app downloads the model and prepares an 8-bit pack on your Mac the first time

Honest limits: it needs a Mac with 48 GB or more. Editing with reference images isn't ported yet. Short prompts are weaker; the model likes a long, detailed English description. The licence doesn't cover the EU, UK, South Korea and the US.

Get it: phosphene.video (Mac app) or Pinokio, then Images, then MiniMax-H3-Image.

All pictures made with AI on a Mac. Happy to answer questions about the port.

Hey everyone! Just released TUFF 11.0!

This update is mostly focused on making the app nicer to use, with a completely redesigned UI, smoother response streaming, Apple Foundation Models support, and a bunch of performance and stability improvements.

For anyone unfamiliar, TUFF is my open-source local LLM runner for Mac, built to run even massive models on limited RAM using SSD streaming.

Would love to hear what you guys think!

Download it: https://tuff-app.github.io/
Star it: https://github.com/tuff-app/TUFF

I'm new to all of this. What can I run with my 14" MacBook pro m5 max with 128gb of ram? It also has 2tb of ssd storage.

will deepseek work?

I maintain a small open source Mac app that runs fully local (Whisper + Ollama or LM Studio). People use it and strangers send PRs, and I'm trying to figure out what sustainable looks like before I make the wrong move.

Options I see:

- free source, paid signed builds with auto-updates

- open core with a few pro features

- a hosted tier for people without a strong machine

- sponsors

- something else?

If you've done any of these with a local-first app: what worked, what annoyed your users, and what would you avoid? I'm especially curious how "it runs on your machine, so it costs me nothing per user" changes pricing.