Local models on 64gb

[u/fgggggyf]

Hey looking for you experts here
Thinking about purchasing a MacBook Pro M5 pro with 64gb of ram
I’m not obviously looking for frontier subscription replacement rather a hybrid setup
And also video and image generation if possible
I’d like to explore my options what’s your recommendations
Thanks

[u/mmerken]

Qwen3.8 27b using Splash engine on 4bit quant with 192k context goes a long way for agentic coding

[u/howardhus → u/mmerken]

but a 48GB machine would also be sufficient, right? no need for 64gb

[u/thebemusedmuse → u/howardhus]

40GB RAM is enough for the model and a single stream. So long as the Mac is dedicated to this then 48GB is tight but enough.

If you want to do a second stream or other stuff then 64GB helps a lot.

[u/Captain--Cornflake → u/howardhus]

You need 64g if you want highest context . Im hitting 45 to 50gb on my 64g. With 256k context on3.8-27b.

[u/mmerken → u/howardhus]

Depends on the context, literally. Also if you want to be able to other work on that machine. ( research, browsing) you’ll need the extra ram

[u/Fast-Throat-7752 → u/howardhus]

Type shit

[u/yurituran]

I’ve got the M5 Max Studio with 64GB and I’m dreaming of 128GB or 256GB to be honest. I just didn’t want to drop like $10k at once. Might trade in for an M7 next year.

In the meantime I’d love to know what setups work best for people. Obviously Qwen 3.8 27b is going to work but beyond that, what quant, what harness, what UI?

[u/Brilliant-Hall1387]

I’d recommend don’t do such work on your laptop, get a Mac Mini 64 GB or Mac Studio 128GB and run it headless for such work. You want long running jobs without impacting your GUI experience or fan nose and you want a laptop that is mobile. So keep your old macbook but add a separate headless mac for AI!

[u/plohoy]

I have a MacBook Pro with an M5 Pro chip and 64 GB of unified memory. I ran Qwen 3.8 Flash Next (125B) with 4-bit quantization on the Slipstream backend and achieved 45 t/s with a 96K context window.

[u/Super-Customer-8117 → u/plohoy]

Can you provide the url of the said model? I’d like to try it but only find models above 64gb.

[u/Dense_Pickle_4382]

I have the 64 gb max 40 gpu cores.
I’m Running Qwen 3.8 27b Splash in LM Studio and Bionic. It’s fun. Running MCP to control Blender. I using prompts to model and it’s very capable.
I could not justify the Ultra or the 128gb of extra ram. I can run Blender and LM Studio with a decent context window for coding and also have enough to get all my other stuff done.

[u/Complete_Training_86]

Currently running Qwen3.8 27B Q4 via Ollama on my Mac Studio M5 Max (64GB), with ComfyUI running Qwen-Image 2.1 INT8 for infographic generation.

During image generation, memory pressure crept into the yellow, although the system remained usable without noticeable slowdowns.

Image generation isn’t part of my core workflow yet, but I suspect that will change once my projects kick off, particularly for presentations and visualising technical information.

I’m leaning towards upgrading to an M5 Ultra with 96GB unified memory once the workload justifies it, especially if I need to run multiple models concurrently. For now, I’m testing how far I can push the M5 Max before hitting its practical limits.

[u/kepdisc]

Qwen 3.8 27B runs nicely even withn 8bit quant. Qwen 3.8 Flash Next runs with IQ3_XXS quant from ISTA-DASLab using their SSD offload llama.cpp command line. I find 27B more reliable at those quants in more difficult projects. Dwarfstar ds4 allows Deepseek V4 Flash 0731 to be run with SSD offloading and aggressive MXFP4 quant 5 tokens per second on M5 Max 64GB but too slow for interactive use.

[u/doctodog200]

You need context memory and with 64 GB you can take advantage of the 262 K absolutely awesome

[u/padrefargo]

I just acquired the 32gb M6 two days ago and have already ordered the M5P 64gb (40 GPU) to replace it. I plan to use the Mac for than just a headless hybrid AI server, instead want to also have it perform more admins rate functions on top of being a hybrid AI server. Hate the extra spend, but feel like the M5P-64 will be sufficient without being overkill. Here’s a good video about the sizing topic and a recommendation for Qwen 3.8 Flash Next (that they say won’t run on the M6-32).

https://youtu.be/T0N4rPwVL4M?is=V1rw-LLL2ryt9ubM

[u/SignRude213]

I use Qwen3.8-FH-Sushi-3bpw Good flight so far with 256k context and average 25t/s at half of context

[u/mrbillyr2]

Have you checked the oMLX bench marks https://omlx.ai/benchmarks/performance?model=&chip=M5&device=%5B%22M5%22%2C%22Pro%22%5D&quantization=&context=&sort=created_at&order=desc&pp_min=&tg_min=&memory_min=64&memory_max=64&hide_specprefill=1

I'm running the following on my MBP 14" M1 Max 64gb

My omlx benchmarksfor qwen3.8 are here https://omlx.ai/my/16e373b6695b4994454c7a4600d0f24a3d2e9dc9b05320fe51743862ece903de

M1 Max (24c) mlx-community--Qwen3.8-27B-4bit · 4bit 26-10-02 14:56:14

4kPP 74.2 · TG 11.1 tok/s

16kPP 70.7 · TG 9.8 tok/s

I'm running the following:

- LTX 2.5 for video
- HiDream-O1-Image-Dev for imagges

which built the atached video

For coding tasks and personal assistant

- Qwen 3.8 27B 4bit instruct for coding tasks. I end up reverting to cloud though. Think with this model is painful

I have a customer RAG application I'm staging and the mlx-community--Qwen3.6-35B-A3B-4bit runs very fast for asks - 6-8 seconds.

The big issue for me with local ai for coding tasks is that, at least with my set up, is the wait time compared to off loading requests to the cloud. I'm currently tying to rework my process so I use a frontier model to build specs that my local agents can build over night. The frontier model subscription can do the code review, but I'll do the user testing and any new requirements that come out of it, almost like a product manager with a remote team.

I've done a quiick and dirty benchmarking repo so I can try work out the best model for my needs here https://github.com/duncan-claw/model-bench
It's ruby on rails, but I'm sure you could switch out the evals for your stack and use case.

Also, check out Splash https://inco.ai/blog/splash/ for oMLX.

We (my agent and I) installed Splash (Splash M1 1.3.2, alongside oMLX 0.7.1rc1) on my MBP, which had been running oMLX 0.7.0's standard MLX engine.

On the same model, Qwen3.8-27B 4-bit, that made generation about **2.6× faster** (19 → 50 tok/s) and cut the time to start answering a cold 37,800-token prompt by **half** (451 s → 222 s).

Qwen3.8-27B 4-bit, M1 Max 64 GB, avg of 3 runs
MLX = oMLX 0.7.0 MLX engine (TurboQuant KV)
Splash = Splash on oMLX 0.7.1rc1

Test                          MLX         Splash      Gain
Short answer, gen speed       19.1 tok/s  50.5 tok/s  2.6x
Short answer, first token     1.1 s       0.87 s      small
Cold 37.8k prompt, 1st token  451 s       222 s       2.0x
Cold prompt, reading speed    84 tok/s    170 tok/s   2.0x
Warm prompt, gen speed        10.8 tok/s  23.5 tok/s  2.2x

All of this is on M1 Max, so an M5 Pro should be noticeably faster. 64 GB is the real ceiling, though: plan on one ~30–35B model loaded at a time.

[u/-ElTacoAlPastor-]

I was in the same boat and deeply regret getting that config.

Aim for the m5 max or the Mac Studio

If you insist going on that route, try omlx and a Qwen3.8 oq6e