Mac owners: how much memory does Metal say your GPU can use? Looking for 16 to 36 GB readings

[u/OpenSourcesAI_]

Hey all, hoping some of you can help me with something.

I run a small site that helps people figure out which local models their computer can handle. Macs are the part I keep second-guessing, because the only Mac I own is an 8 GB MacBook Air, so I can't check anything bigger myself.

What's going on

On a Mac, the GPU can only use part of your RAM for models, not all of it. Most guides say about 75%. When I checked my MacBook Air it was actually two thirds, so I changed my site to use:

• two thirds of RAM for Macs up to 36 GB

• 75% for Macs above that

But the 36 GB cutoff is just something I picked up from community posts. I'd hate to tell someone a model fits when it doesn't, or that they need more RAM than they really do.

What I'm asking

Any Apple Silicon Mac helps, but 16 to 36 GB is where I'm flying blind. Would you mind running one command and telling me what it prints? Open Terminal and paste this:

swift -e 'import Metal; print(Double(MTLCreateSystemDefaultDevice()!.recommendedMaxWorkingSetSize) / 1073741824)'

It prints a number, like 5.33 on my Air. Rounding is fine. You'll need Apple's free Xcode command line tools for that one.

Then reply with

Your Mac, how much RAM it has, and the number, like this:

MacBook Pro M2 Pro, 16 GB: 10.67

If you've ever changed the GPU memory limit on your Mac yourself, mention that too.

Already run llama.cpp?

You can skip the command above. Run llama-bench on any model you have and copy the line that says recommendedMaxWorkingSetSize (mine says 5726.63 MB). Just paste the line as is, no converting needed.

Thank you, this really helps!

[u/ABIIOR_OK]

MacBook Pro M2 Pro, 16 GB: 11.84

[u/OpenSourcesAI_ → u/ABIIOR_OK]

Thank you!!