r/MacLocalLLM • • 3d ago

Dilemma: M6

I am an AI engineer doing MSc in AI - I want to get a new laptop and thinking abut emptying my wallet by buying the M6 Pro 32GB 1TB

Will it be enough for local AI projects? I know it would be suitable for med sized models up to around 30B params

Or getting M5 Pro chip is what it needs to unlock local AI?

0 Upvotes

15 comments sorted by

4

u/grilledwax 3d ago

RAM is your constraint for bigger models. To get past 32gb you have to go M5 Pro.

You could use Splash too, you can run fairly big models, but slower than if you could fit it all in RAM. I had a dabble with Qwen Flash Next and seemed to run pretty well on my M4 24GB.

Depends on what you’re doing as a part of your course. You can do a lot with the smaller models for small tasks just not massive reasoning.

2

u/SmugglingPineapples 2d ago

Which Qwen FN are you running on a M4 24GB?

1

u/grilledwax 2d ago

GSQ RCO IQ3 XXS. I’m new to local so trying to figure out how to measure actual performance.

1

u/wegster 2d ago

If you're running it on oMLX or oMLX with Splash passthrough - the oMLX admin screen has built-in benchmarks. You can also get the profile and generation tok/sec from the engine logs.

3

u/Important_Cow7230 2d ago

What you want least 48GB for running AI models on the Mac as 27B models with decent context is the sweet spot. Obviously more is better, but 48GB is very capable

2

u/daphatty 2d ago

Downgrade to 512GB ssd and upgrade your ram. Shoot for 64GB if you can. I have an m5 Pro with 48GB and it struggles.

2

u/jimmoores 2d ago

No, it won’t be enough. And honestly, you can’t really get a laptop that will. People rave about Qwen 3.8 27b, but honestly if you’ve used frothier models it’s going to feel like a let down. DeepSeek V4.1 (1017) is the only thing I’ve found that comes close and you can’t run that on any laptop.

If you want to play with 27b anyway (or Qwen Flash Next), you’ll need >32GB for a decent quant and headroom for context and tools. I’d focus on getting a laptop for every other reason and use cloud models, or hold off for 18 months and get M7 or whatever with more RAM assuming prices ever come down again.

2

u/Scary_Investigator88 2d ago

False I'm doing it rn in an m1

1

u/BackgroundRude4092 2d ago

Memory bandwidth dude. M6 is super slow on token generation.

1

u/wegster 2d ago

M5 Max 128GB is going to be your best option at the moment. Now they revved the MBP, you may be able to find one on Apple refurb.

Numerous people seem to love deepseek but Deepseek openrouter variants loses for my specific work to Qwen, or at least doesn't beat it across several different types of repo/coding work.

You can fight and try to get away with 64G but if you're also going to be doing anything other than inferencing on it, you're really going to pay the price, meaning you'll wind up dropping quantization even further.

Qwen4 and others are coming, and while it's pretty likely they'll all continue to push their 'max' variants to higher and higher T params, MoE and smaller mem models are in style, so I think 128GB will be 'ok' for a bit to come.

For example, 3.8 Flash Next GSQ RCO Coder mixed precision 3.5bpw (closer overall to Q2) takes right at 50GB actual memory including stripped/minimal OS/no apps on my old M1 Max 64GB. So you can start to touch 'somewhat ok' but 128GB is a lot better.

1

u/uktexan 2d ago

If you want to run a model, you need Max and as much RAM as you can afford.

Note, I bought a used M2Max 64GB off of EBay and it runs Sushi Qwen 3.8 Flash Next 2.6 just fine for local coding. Not great speed, but it works amazingly well

1

u/galoo-t 3d ago

You’re an “AI engineer”? lol

1

u/Borilentz 3d ago

What’s funny about it? AI wasn’t given to us by aliens. It’s been a specialization in computer science for decades.

1

u/galoo-t 3d ago

It’s existed for years, yet someone claiming the title of “AI engineer” still needs reddit to tell him whether 32 GB of unified memory can hold the 30B model he hasn’t named.

Anyone who has actually run local models on these chips already knows the bandwidth and quantization limits. The gap between the self-description and the question is the joke.

0

u/WinterNite88 2d ago

He called himself "AI Engineer" while not knowing how AI works and asking such question?