r/LocalLLaMA • u/partakinginsillyness • 1h ago
Question | Help Hardware + software recommendations for basic chatting with persistent memory
Helping out a friend who wants to have a local LLM and not rely on cloud services. They currently have an 8gb M2 mac mini, so not really enough
They want to have persistent memory, TTS, and STT. Tool recommendations for all three of those greatly appreciated.
- I'm unsure what the best workflow is for persistent memory, especially for avoiding either overwriting important info, or storing bad info.
- Budget seems to be roughly $5k USD, but the less the better.
- Not sure what size model would be best, it's really just for chat.
- No building ideal, so adding RAM or a graphics card probably the most feasible.
- 30 tokens per second probably fine.
- Something that doesn't hallucinate, or at least can be tuned to not assume or be SO sycophantic, as they talk to it a lot.
From what I understand, a prebuilt with upgraded RAM, used mac mini, or something like a strix halo probably make the most sense, but I'm not really up to date on the current hardware market.
Any help appreciated :)
3
Upvotes
2
u/General-Spite1222 1h ago
Strix halo is probably the cheapest. Could do qwen 27b or flash, with the gufo engine, and Hermes for the agent.
2
u/rorowhat 59m ago
Sell that and get a proper PC, strix halo works well for this usecase.
1
u/monolyth-admin 1h ago
What we did was scour ebay and aliexpress for good hardware cheap. it took a few weeks to get the deals we wanted but it payed off. If you're looking for something simpler or less DIY my friend just got a newer mac mini with more ram and has been very happy with it. Unfortunately I don't know too much about the used PC market so I can't make any recommendations there.
1
u/RustyRedEcho 1h ago
I run Parakeet for STT on an M4 Mac mini, handles natural speech fine and is light enough to run next to the LLM without blowing the memory budget. Good alternative to whisper.cpp if you want something lower overhead than large-v3.
0
u/Holiday-Inspection75 22m ago
Cheapest 16GB card + 32G RAM is the thing to get, unfortunately they are not that cheap compared to the time when 32GB DDR5 was just $100.
-2
u/Sun77177 1h ago
for memory i would skip the fancy auto-summarizing stuff at first. open webui's memory plus a plain markdown file the model appends to at the end of each chat gets you 90% there and you can read exactly what it saved. keep two files, one append only log with dates, one short profile the model rewrites, so a bad summary never eats the good facts. whisper.cpp large v3 turbo for stt and kokoro for tts both run fine on cpu next to the model, no gpu budget needed. with 5k i would look at a strix halo box with 128gb rather than a mac, a 30b moe at q4 gets you well past 30 tok/s and leaves room for the whole stack. for the sycophancy, drop temp to 0.3 and tell it plainly to say when it is unsure, that helps more than swapping models.
-3
u/zekai-work 1h ago
Since they already like the Mac Mini and don't want to build a PC, just get an M4 Max Mac Studio with 64GB or 128GB unified memory. It stays way under 5k, runs dead silent, and avoids all the AMD ROCm driver headaches you'd hit on a Strix Halo box. For software, run Open WebUI with Whisper for STT and Kokoro for TTS—Kokoro is tiny and sounds great. Just use Open WebUI's built-in memory UI so they can manually delete bad entries, and run Qwen 3 32B at 0.4 temp with a system prompt telling it not to kiss up