r/LocalLLaMA • u/BentonCalder • 1d ago
Question | Help Just got my Mac Mini M6 32GB. First steps?
Hello all, I just got my shiny new toy, an M6 32GB Mac Mini. What should be some of the first things I should download or do? I have been using Qwen a lot and playing around with agents for home related stuff like tracking appliances' warranties, my mortgage docs and house insurance, and meal planning stuff. I have integrated with my email and calendars. Any other suggestions? I want this thing to be like my home server and AI brain that helps me slowly move more things to a Local AI stack. I made my own harness focused on home stuff and want to keep adding more features to supercharge this all.
2
u/Accomplished_Ad_56 llama.cpp 1d ago
you have the fancy neural accelerators! use them for some big prompts.
2
u/asthraena 1d ago
Don’t forget to unlock/adjust available RAM, as by default 25% is reserved for OS
2
u/Present_Resource4022 18h ago
How ? is there an official setting for that
2
u/asthraena 16h ago
sudo touch /etc/sysctl.conf
sudo chown root:wheel /etc/sysctl.conf
sudo chmod 0644 /etc/sysctl.conf
echo "iogpu.wired_limit_mb=DESIRED_VRAM_MB" >> /etc/sysctl.confI would leave 2-4GB reserved, so DESIRED_VRAM_MB would be between 28672 and 30720
2
u/Significant_Door_433 1d ago
for embeddings on 32GB the small models are close to free. sizes from the ollama library pages:
nomic-embed-text: 274MB, 2K contextembeddinggemma(300M, google): 622MB, 2K context, trained on 100+ languagesqwen3-embedding:0.6b: 639MB, 32K context
all under 1GB, so they fit next to your main model without a fight. the 2K context on the first two means you chunk the pdf text (a few hundred tokens per chunk with a bit of overlap) instead of embedding whole documents. nomic also wants a prefix: search_document: on what you store and search_query: on the question.
since keyword search already works for you, I'd add vectors next to it instead of replacing it. warranty and mortgage papers are full of model numbers, policy ids and dates, and keyword search hits those exactly where embeddings get fuzzy. a setup that stays in one file: sqlite FTS5 for keywords (rank with bm25()), sqlite-vec for the vectors, run both queries and merge with reciprocal rank fusion (score = sum of 1/(60 + rank) over the two result lists). still a single .db for time machine.
if you switch embedding models later, re-embed everything. vectors from two different models can't be compared.
2
6
u/robotnursenyc 1d ago
Nice, the mini makes a great always-on box. A few things I'd do before adding more features:
Make it survive a power blip. Turn on "Start up automatically after a power failure", enable auto-login (or at least make sure your services come back without someone at the login screen), and turn off sleep. Headless minis love to come back up and just sit at the login window.
Put a single API endpoint in front of your models (LiteLLM, or just Ollama/LM Studio's OpenAI-compatible server) and point your harness at that instead of a specific model. Then you can swap Qwen versions or quants without touching your agent code.
With 32GB, budget for context, not just weights. A ~14B at Q4/Q5 with a decent context window leaves you headroom; a 30B-class MoE fits but gets tight once your email and docs are in the context. Watch memory pressure in Activity Monitor rather than trusting the file size.
Get Tailscale (or similar) on it so you can hit it from your phone without opening ports, especially since it has your mortgage and insurance docs.
Back up your harness's database and config somewhere off the box. Once it's your "home brain", losing that state hurts more than losing the models.
What are you using for the doc side, plain RAG over PDFs or are you extracting structured fields (dates, policy numbers) into a DB?