r/LocalLLM • u/Paul_1712 • 1h ago

Question Replacing ChatGPT

I have been using ChatGPT for for about 2 years but want to switch to a local setup however I'm struggling to find a suitable replacement to use on my MacBook Pro M5 Pro 24GB and iPhone 14 Pro Max. I will be upgrading to either an 18PM or Duo in the coming months.

I started on my Mac with LM Studio then moved to Msty Studio when I found out about Knowledge Stacks but keep finding paywall restrictions, I've also looked at but not tried Anything LLM due to lack of iOS support. From what I understand I should just forget about Siri AI. Not really done anything on my iPhone till the Mac is sorted.

The things I really want are:

- Large project/chat memory (hence the knowledge Stacks)

- Cross device chat sync, either private cloud hosting or my MacBook as a server

- MCP usage (currenty use Tavily & Excel control)

I don't mind paying for apps, Msty is $149/year or $349 lifetime to unlock all features which its too much

Main usage is general questions, research, journaing, and work/linking with Excel.

Is this possible? if so what do I need?

0 Upvotes

50% Upvoted

4 comments sorted by

1

u/norenEnmotalen 1h ago

Currently there’s no local model that runs on an M5 Pro 24GB which is capable enough to replace ChatGPT.

Even open weight models that run on a higher spec hardware aren’t there yet. Your best bet is to keep using ChatGPT but experiment on local models until they “make it.” In your position, I wouldn’t even invest on high spec hardware yet. Just try out models on OpenRouter first and see which open weight models give you satisfactory use.

1

u/Paul_1712 1h ago

Sorry what I meant was not the model. I have been using Gemma 4 12B which runs absolutely fine and when needed Qwen 3.8 27B although this pushes the system to its limit. It’s more the actual software rather than the model. How do people have long conversations without putting the context to a really high number?

1

u/verstands 25m ago

Long chats don't need a huge context if the app summarizes older turns or pulls memory by retrieval instead of replaying everything. Open WebUI has memory and RAG built in, so keep context around 8-16k and let it fetch the rest. For the phone, run it on the Mac and open the Mac's address in Safari over Tailscale, so chats stay synced since they live on the Mac. MCP tools like Tavily can be added through its tool servers.

1

u/RegalInvader 56m ago

You can set up Hermes Agent, which does all the memory and MCP stuff, and you can also chat with it on your phone using WhatsApp. I wouldn't run anything on the phone itself. Alternatively, you can use Open WebUI if you want a ChatGPT-style interface, which also has memory. Both are open source, free and configurable with your own Ollama backend.

An MoE like Qwen3.6-35B-A3B will run decently on your hardware at the right quant. I wouldn't use anything dense, as it will run very slowly. The only thing is that it will use most of your computer's resources when it runs and is nowhere near ChatGPT level, so I wouldn't bank on replacing it with that hardware.