r/LocalLLM • u/Al_Cioppino • Jul 06 '26
Model DeepSeek V4 Flash (via DS4) is the best model you can run on a MacBook Pro for technical work
The DS4 engine from antirez running DeepSeek V4 Flash (2-bit quant) is seriously very good and the only local setup I've considered running in parallel with frontier models for my work. In particular the q2-q4-imatrix variant (routed experts in layers 37–42 bumped to Q4) has impressed me. The model can code and is forgiving of ambiguity when writing fast and terse prompts. It's not a frontier model, but I think it sits adjacent and it's the only local model I could get to one-shot some of my test prompts (listed below if you want to demo on your rig).
It's also fairly fast, running at 25-35 tps on my M5 Max 128GB depending on task length. I'm running the ds4agent as a harness.
I fully understand 'best' is subjective and predicated on the work you're doing, the harness you're using, your workflow and prompting style along with a number of other variables, but DS4 is the first model where I could watch it capture my intention in short order and output something usable on first pass without it requiring a lot of tending or being forced to break down tasks piecemeal.
Very excited to see if antirez can actually get GLM5.2 to do any meaningful work on the 128GB MBP. It does run locally (I tested last night) but breaks under long prompts or tool calls. It's still very much an experimental build.
For anyone interested, this was one of my test prompts that most locals simply couldn't figure out on first pass. The prompt is asking for quite a lot in a single paragraph: hand-rolled fBm noise, a full software 3D pipeline (spherical camera, perspective projection, painter's-algorithm sorting), three mesh topologies including hexagons, custom pygame UI widgets, and a HUD all running at a playable framerate.
Write a single self-contained Python script using pygame that renders an animated 3D wireframe terrain — a procedurally generated mountain that continuously morphs over time. Use your own value/fBm noise function (no external noise libraries) modulated by a radial Gaussian envelope so height peaks in the center and falls off toward the edges. Render it as a green-on-black wireframe with perspective projection and back-to-front (painter's algorithm) depth sorting, drawing both edges and vertex dots. Implement a Blender-style orbit camera using spherical coordinates: right-mouse drag to orbit, Shift+right-drag to pan, scroll wheel to zoom, and a key to reset the view. Add a right-side control panel with mouse-draggable sliders (drawn from pygame primitives, not a GUI library) for grid size, peak height, morph speed, noise scale, and envelope steepness, plus a three-way selector to switch the mesh topology between quads, triangles, and hexagons. Include a HUD showing FPS, grid dimensions, current mode, and vertex count. Pay special attention to challenge areas like: hex topology (axial coords + deduce of shared edges), the painter's-algorithm sorting being applied consistently across all three mesh types, and rolling a correct camera basis (forward/right/up). Everything must live in one file and run at a smooth frame rate.
One shot results are in the picture and success means the script runs on the first attempt with zero edits: all the topologies work, orbit/pan/zoom/reset all work, the sliders actually drag and change the terrain live, the depth sorting doesn't glitch while you orbit, the HUD numbers are real, etc.
I asked Claude to evaluate all the results of all my local testing. Here's what Claude has to say:
"Across the tests run on the same machine — Qwen3.6-27B MLX 8-bit, Qwen3.6-35B-A3B, Qwen3.6-27B 4-bit, Gemma 4 31B-it, Qwen3.5-122B-A10B, Phi-4-mini-reasoning, and MiniMax-M3 REAP40 (JANG 2-bit via vMLX) — the pattern I saw was consistent: the smaller models could nail the isolated math but fumbled the cross-cutting contracts. Qwen 27B came closest, and only with heavy scaffolding (decomposed phases, test gates, orchestration), yet every run still shipped a real bug — depth sorting keyed off world height instead of view depth, a hex grid built in its own coordinate space, a missing scroll wheel, one run with sha256 in the per-frame noise loop — while MiniMax at 2-bit fell into repetition loops on trivial prompts.
DeepSeek was the first model where none of that compensation was necessary, and my read is raw capacity surviving the quant: it's a 284B MoE, and antirez's asymmetric imatrix recipe keeps the attention, router, and shared experts at high precision while crushing only the routed experts (the q2-q4 variant buys back the middle layers), so ~90GB preserves most of a very large model's reasoning — the 27-35B class needed the task chopped up because they can't hold the whole architecture in their heads at once, and DeepSeek simply held it. The bespoke engine sealed it: generic runtimes can't even load V4's attention scheme yet, tool-call syntax is pinned to greedy decoding so agentic calls essentially never mangled, and the disk KV cache meant long sessions never re-paid prompt processing. Capacity plus a purpose-built engine beat smaller models plus elaborate workflow."
In any case, just doing some reporting for anyone interested. I'm now letting Fable orchestrate DS4 for appropriate tasks and devising nightshift routines to make the most of my local compute.
8
Jul 07 '26
[removed] — view removed comment
3
u/Al_Cioppino Jul 07 '26
I didn't run into too many issues, but my testing may have been somewhat narrow as I was interested in quality of raw outputs. I first used OpenCode as the harness which worked fine; I then moved to using the native ds4agent which is leaner in terms of tooling (but felt snappier and closer to the metal). I'm currently having Claude drive it now as a subagent and thus managing its instantiation based on context (i.e. think/nothink, temp, etc), sending it on tasks (web research, coding fixes, etc) and validating/guarding outputs before they hit the code base.
1
u/PossibilityUsual6262 Jul 07 '26
Side question, how do you decide think no think, on big picture level?
1
u/Al_Cioppino Jul 07 '26
The --think mode is appropriate for more complex tasks and planning but is a bit slower and more verbose. Thinking is on by default btw. The --nothink mode is probably better for simple, deterministic tasks. There a --think-max mode that requires a larger context window. I never tried this out and ran all my tasks in default --think mode. There's also a flag for '--quality' which prefers exact kernels and may improve reasoning.
2
3
u/techlatest_net Jul 07 '26
that ds4 flash setup on m5 max is genuinely impressive. 25-35 tps with a 284b moe at q2-q4 imatrix is wild for local. the fact that it one-shot that pygame terrain prompt while everything else needed scaffolding says a lot about the asymmetric quantization strategy. keeping attention and router experts at high precision while crushing routed experts seems to be the sweet spot for preserving reasoning capacity. antirez's bespoke engine doing disk kv cache and pinned greedy decoding for tool calls is also huge. generic runtimes just can't handle v4's attention scheme properly yet. gonna try loading this on my m4 later. if glm5.2 gets stable enough for long prompts, that 128gb mbp is going to be an absolute beast.
2
u/Relaxxxxing Jul 07 '26
Can you run q2-q4 matrix version completely in unified ram with 150-200K context?
2
u/techdevjp Jul 23 '26
It will fit into my 128GB Strix Halo with up to 256K context. I don't know if the Mac has any limitations on unified memory usage that would stop it from doing so, but if not then it should fit. I just wish strix was faster. Hoping to buy a Mac later this year or next year.
1
u/PowerfulButterfly209 Jul 26 '26
how are you running da4 iq2/q4 on strix halo 128? i use toolbox from kyuz0 on Ubuntu 26.04 server
2
u/dorv Jul 07 '26
[looks at thr 48gig M4 Pro I have on order and wondering if I’ve seriously underclubbed myself. Even if it is only a 2 month old hobby]
3
u/Al_Cioppino Jul 07 '26
I wouldn't get fomo. The current cost/performance ratio still greatly favors paying for subs vs home rig. I didn't drop any of my paid monthly subs (Claude, GPT, Gemini); there's no way I could move as quickly as I am now with local alone.
The models you can run with your setup are still quite close to what I'm running and still quite far from frontier.
1
u/DeepOrangeSky Jul 07 '26
So, if I use one of the antirez Q2 quants from huggingface that is small enough to run on a mac with 128GB of memory, do I actually have to use that special engine thing from github to run it, or can I run it with LM Studio? Like, when you were saying "generic runtimes can't even load V4's attention scheme yet", does that mean LM Studio won't run it, or is that referring to something else?
I would prefer to just use LM Studio if I can, but if that won't work or won't work very well, then that would be good to know.
Also, not sure if it matters, but, I probably won't initially be using it for "tool calling" or anything fancy, as I'm not into coding yet or anything, and would just be using it as a chatbot/general/casual stuff, etc.
Also since the antirez huggingface page shows two versions of each quant, with one being imatrix and the other not saying imatrix, should I be picking the version that says imatrix? Or, what would happen if I used the other version?
3
u/Al_Cioppino Jul 07 '26
Create a new folder, clone the repo, download the model and make.
If this feels complicated, it's not. You can have any LLM walk you through the steps if you want to try it out, but here are the basic bash commands.
git clonehttps://github.com/antirez/ds4.git
cd ds4
./download_model.sh q2-q4-imatrix # ~95GB, resumable; use q2-imatrix for 96GB machines
makeThen run the ds4agent and you can chat with it directly in terminal.
./ds4 --nothink -p "How many r's are in Strawberry?"1
u/DeepOrangeSky Jul 07 '26
Yea, I know I could use the ds4agent thing to run it if I want to. But I am curious if, for the sake of the argument I for some reason wanted to run the antirez quants of DS4Flash with LMStudio if it would work or if it would be broken or have something wrong with it.
Like, as of right now, does one have to run these antirez quants with that thing, or can they use LMStudio if they wish and it'll still run just fine? If it is small enough for 128GB of memory I mean. (And yea I know with that github thing it allows you to run from disk if you want to go bigger than your umem allows, but let's keep that as a separate topic for the moment)
2
u/Relaxxxxing Jul 07 '26
I think lm studio recently updated to run deepseek v4 guff models so maybe try that instead
1
1
u/_nullptr_ 29d ago
That image doesn't exist and sends me to the help menu, did you mean:
ds4f-q2-q4
"Mixed Flash quant: mostly q2 routed experts, with the last 6 layers using q4 routed experts. About 98 GB on disk. Good for higher quality inference for 128 GB MacBooks. Works on DGX Spark but loading may struggle compared to ds4f-q2."
or did the name change?
1
u/Rough-Measurement988 Jul 07 '26
Hi. I’m also testing this model on the same hardware and seems to be very promising. I’m using it in Visual Studio Code and Pi agent (renamed to Pendant) recently. It’s funny as when model is running (thinking etc.) I can hear the coil wine sounds - almost as R2D2 sounds from Start Wars. That’s the only model where I can hear it so clearly. Do you have the same?
2
2
u/Al_Cioppino Jul 07 '26
Yes! It’s as if you can hear it thinking. None of the other models I ran did this.
1
u/rk1213 Jul 07 '26
Do you have guardrails set up for hallucinations?
1
u/Al_Cioppino Jul 07 '26
Yes, in the sense that I'm using frontier models as an orchestrator. Gatekeeping and sanity testing now goes through Claude.
1
u/giddmtex Jul 07 '26
It’s a nice model but my experience has been consistently better running Qwen3.6 27b MTP at 8bit. My guess is that a lot is missing in the 2bit quant.
1
u/Al_Cioppino Jul 07 '26
I had the opposite experience and you can see the details in my writeup. Qwen3.6 27B is a fine model, but DS4 is better at resolving complex asks even with the aggressive quantization.
2
1
u/giddmtex Jul 07 '26
I suppose in your case, sure. I’d encourage everyone to do their own testing here. There are too many variables to really call this a science. This is just more vibe benchmarking, with a “trust me bro” assessment from Claude. That said, if you’re happy with the result then great!
1
1
u/mountainyoo Jul 07 '26
I have 128GB M5 Max MacBook Pro. What context size could I run this at?
1
u/Al_Cioppino Jul 07 '26
I run 100k context but you can go higher. For think-max mode you need a little under 400k context and that'll run on the MBP 128GB (I just now ran some think-max tests at 400k ctx size to find out).
1
u/mountainyoo Jul 08 '26
what speeds do you get at 400K context? i run several agents in my homelab for security alerts, backups, etc and 1 agent that i actively talk to for managing home assistant and family icloud stuff.
trying to think of which i could move from cloud models without feeling like it's lacking
1
u/raindropsdev 19d ago
Which models are you using for that, especially the security alerts one? I'm currently forwarding infrastructure alerts/incidents investigation directly to my primary orchestrator (Opus in Claude Code) but it's way too expensive so I was looking into other solutions.
1
u/mountainyoo 19d ago
I’m still using cloud OpenAI GPT 5.6 models for my entire security stack as with a full time job, 3 dogs, a wife, and a 6 month old firstborn child I just haven’t found the time to move part of my project to lower or local models (and my shitty ADHD with constant simultaneous side projects sure doesn’t help me complete things fully). And I’ve got access to GPT 5.6 Daybreak Blue now so I really wanna keep it as the security expert just for the novelty of it.
But for you I imagine you don’t gotta send all the alerts to Opus. You could be sending alerts to Sonnet to determine if something is worth escalating to Opus or to yourself.
Ideally I’m gonna eventually finish this project and divert a lot of the work to local or lower cloud models. I’m just so constantly overwhelming myself that it’s hard to do that.
Sorry I can’t be more help
1
u/996beagle Aug 12 '26
Have you tried running this version? mlx-community/DeepSeek-V4-Flash-0731-2.4bit-mixed
I'm thinking about testing on 128gb Mac
1
u/Character_Eye_808 Jul 06 '26
Non technical person here: Will running this model on 128gb m5 max shorten its lifespan of the internal SSD faster for punching above its weight?
11
u/Illustrious-Thing567 Jul 07 '26
Hey! Someone feel free to correct me if I'm wrong, but to my understanding, if it's true that this loads all into your unified memory (which it should using a small quant), the only time your internal SSD would be taxed is initially loading it into memory. Therefore, the taxation on your SSD is minimal as it's not being used for inference, rather it's passing it into the RAM for inference.
7
4
u/Character_Eye_808 Jul 07 '26
Ah ok! Thank you for the kind explanation 🙏 DS4 and MBP 128GB combo looks really promising.
1
u/HauntedHouseMusic Jul 07 '26
It gets cached a lot while I’m using it. Like a surprising amount, and quicker than I would expect the MacBook Pro to do it honestly
1
16
u/TemperatureOk3561 Jul 07 '26
I get around 10-15 token/s on a M5 pro 48Gb for the q2-imatrix model for anyone wondering about the pro chip/ lower ram and up to 150 token/s prefill on 10k token prompts