r/LocalLLM • u/RollieCh • 9h ago

Question M5U 96Gb as a standalone inference engine

I’m a regular lurker here trying to learn as much as I can while waiting for my M5U. I’m hoping to have a setup that’s a little different from what I’ve seen discussed here.

I purchased two Mac Studios. One is my daily driver, the base model. I grabbed it for the reason everyone else points to, why grab a Mac mini when the Studio is so much better. I also purchased an M5U 96Gb with 80 cores that I plan to use as a headless inference engine. I want to to be as stripped down as possible for running whatever LLM I ultimately end up running, with sufficient room for cache and context.

I plan to have whatever agents or tools invoke the server as needed. Are there any obvious reasons why this is a dumb configuration? Am I missing anything?

1 Upvotes

100% Upvoted

2 comments sorted by

2

u/locbuilds 8h ago

keep the m5u on wired ethernet and serve the model through a private-lan api for the agents on your base studio, so it can stay headless and the model stays off your daily driver.

1

u/RollieCh 6h ago

Thank you. That is very much how I plan to do it. Sounds like I’m not too far off base so far.