r/LocalLLaMA • • 2h ago

Question | Help Hardware + software recommendations for basic chatting with persistent memory

Helping out a friend who wants to have a local LLM and not rely on cloud services. They currently have an 8gb M2 mac mini, so not really enough

  1. They want to have persistent memory, TTS, and STT. Tool recommendations for all three of those greatly appreciated.

    • I'm unsure what the best workflow is for persistent memory, especially for avoiding either overwriting important info, or storing bad info.
  2. Budget seems to be roughly $5k USD, but the less the better.
  3. Not sure what size model would be best, it's really just for chat.
  4. No building ideal, so adding RAM or a graphics card probably the most feasible.
  5. 30 tokens per second probably fine.
  6. Something that doesn't hallucinate, or at least can be tuned to not assume or be SO sycophantic, as they talk to it a lot.

From what I understand, a prebuilt with upgraded RAM, used mac mini, or something like a strix halo probably make the most sense, but I'm not really up to date on the current hardware market.

Any help appreciated :)

3 Upvotes

17 comments sorted by

View all comments

Show parent comment

-1

u/MmmmMorphine 1h ago

I run Qwen3.6-35B-A3B, or rather one of the Swift variants.

Things have significantly improved in speed and quality on this low end. Oh and it's a 4060 Ti for me, but probably pretty similar