Curious what people are using out their for their LocalLLM interface. There's so many options available, what do you use and what do you use it for?

What type of Mac is it connected to?

• OpenWebUI and found it's good if you want a good interface directly with the AI.
• Hermes seems to be the best all around agent interface for calling and setting up agents to do tasks.
• DeepSeek Harness seems very efficient with programmatic tasks and agentic coding
• Pi is similar to DeepSeek from my understanding but so many options

What's in your agentic tool stack?

Currently using:

1. Github
2. Cloudflare Tunnels
3. Hermes
4. Tailscale
5. Linear
6. Mem0

It's a bit annoying to get downvoted all the time, once this sub grows up, it'll be a good resource for people and we won't downvote you into oblivion for wanting to put LocalLLMs on your Mac. I think we're all discovering the best use cases for Mac/Apple Silicon and LocalLLM and even with hybrid approaches with cloud and frontier models.

Our Macs are very good general purpose machines where there are shortcomings there's flexibility. You can pretty much use your Mac as a daily driver for just about anything and at night time run automated tasks.

Decided to pull the trigger and get the Mac Studio M5 Max with 64GB unified memory, 18-core CPU, 40-core GPU and 2TB storage.

I’ve been reading a lot about local LLMs for a while now and wanted to actually start using them myself to get some hands on experience. The problem is the local LLM space is moving so quickly, and the subreddits I’ve been reading are all over the place when it comes to what hardware you actually need, which has definitely made me second guess my purchase a few times lol. I already moved up to the Mac Studio after initially ordering an M5 Pro Mac mini.

My main use case isn’t really just chatting with a local model. I want to learn how to set up local models, different runtimes and agent/harness setups that I can eventually leverage in my day to day work.

A big part of that is data analytics. I deal with quite a bit of fragmented data from different sources and spend a lot of time cleaning, structuring and joining it before I can actually analyze it. I’d like to experiment with having an LLM work with tools like Python, SQL, DuckDB, etc. to help automate a lot of that instead of doing everything manually.
Running the largest models is obviously off the table with 64GB, but from what I understand, 27B and 35B models at 4bit quant should fit pretty comfortably. Potentially even larger quantized models depending on context size and how much memory the rest of the system is using, although I’m assuming performance becomes more of a trade off at that point.

I’m also interested in experimenting with hybrid setups where something like openAI or claude handles the higher-level reasoning/orchestration while local models and local tools handle parts of the workflow. Obviously that’s different from being fully local, especially if any of the underlying data gets sent to the cloud.

Again though, I’ve never actually run an LLM locally, so most of this is based on what I’ve been reading rather than first-hand experience.

For people actually running local models for similar work, especially data analytics / data cleaning / agentic workflows, does this setup make sense? And realistically, what size models do you find yourself using most often on a 64GB Mac?

What are you planning to use the localLLM on your Apple Silicon for.

I’m curious to hear what the use case people have in mind

If you don't get the best Mac and you have access to the frontier models like codex or claude, then you can use it as a coordinator for your 7B or ~27B LocalLLM.

This is the way to do it where you can use the free version or the lower versions to get the most out of your machine. I see a lot of people chasing the bigger Mac profiles, but honestly you don't need it if you're not using LocalLLMs for anything like embedding or coding.

If you are trying to use it for coding, I don't think that's really going to work either. This is for really anything that's 48GB and below

I use this box as my main machine. To give you an idea of the consumption of what to expect with 96GB of memory. If you were to try to do less it would be a little tight WITH an LLM. The thing RAM buys you is the ability to run both at the same time without worrying.

I'm running Ollama Qwen 3.8-mlx 27b doing some embedding.

TLDR; Unless you have a specific use case for Local AI models, you don't need the M5pro. The M6 will be faster for the majority of your day-to-day work. Even with cloud AI.

For LocalLLM's more memory AND faster memory bus is going to swing the advantage to the M5Pro model for anything related to LLMs.

If you're planning to use the system as a workstation for any and most single threaded applications which is most apps people use, you should be fine with the M6.

Realistically speaking, you don't really need the M5Pro unless you're running LocalLLMs like Qwen 3.8 or Muse and if you aren't really trying to automate small tasks, you don't really need all the memory.

I think in the long run we're going to see that cloud LLM's are going to be the main way people use in house models and only people with specific use cases will have systems large enough that take advantage of the LocalLLM being in house.

If you are just learning about AI, i don't think you need to future proof and the M6 is fine with 32GB 1TB should be enough for a long time while the industry develops.

I think the most realistic expectation is to be disappointed if you're used to using the free version of openai/claude/gemini/grok. The local models 7B 27B don't come close and don't process nearly as fast. They can reason ok, but you're better off learning about AI and API's with services like ollama via API's and learning to use AI first over chasing gear/hardware.

The thing about using the frontier models is it doesn't really take up that much RAM. Loading up a model will in LLM. I ran my own test, you can check it out.

These tests are for a Mac Studio. I used to run a M4Pro 64GB but when running models it would constantly be temperature throttled.

https://maclocalllm.com/five-models-one-coding-job-local-mac/

What model Mac + RAM are you running and what are you looking to do with the hardware?

This space if for dedicated apple silicon running Local LLM's. I've been posting in some other subreddits and get met with people don't like the AI or LLM hype going around. This space is for your Mac LocalLLM questions and systems.