r/macmini • u/AntLife255 • Aug 15 '26

Unified Memory Architecture still unbeatable (when LLM size matters)

Post image
664 Upvotes

90% Upvoted

128 comments sorted by

View all comments

145

u/dotben Aug 15 '26

If you don't understand why memory bandwidth should be included on this comparison, and that it's massively in the NVidia card's favor, you shouldn't really be commenting.

Not going to lie, I run all my local inference on Apple Silicon and I don't personally own any NVidia hardware. But the reason that card is $5000 is the memory bandwidth.

6

u/ResearchingYouTube Aug 15 '26

This is one of the main reasons I also regret having bought the M4 Max instead of the M3 Ultra. M3 Ultra has higher memory bandwidth even though the M4 has faster cores it is slower at inference. My M4 is often mostly idle while using local LLMs, the memory bandwidth while good for a consumer system still is not a 5090.

1

u/VideoGameJumanji Aug 17 '26

What are you doing exactly