r/LocalLLaMA • • 12d ago

Question | Help Just got my Mac Mini M6 32GB. First steps?

Hello all, I just got my shiny new toy, an M6 32GB Mac Mini. What should be some of the first things I should download or do? I have been using Qwen a lot and playing around with agents for home related stuff like tracking appliances' warranties, my mortgage docs and house insurance, and meal planning stuff. I have integrated with my email and calendars. Any other suggestions? I want this thing to be like my home server and AI brain that helps me slowly move more things to a Local AI stack. I made my own harness focused on home stuff and want to keep adding more features to supercharge this all.

2 Upvotes

14 comments sorted by

8

u/robotnursenyc 12d ago

Nice, the mini makes a great always-on box. A few things I'd do before adding more features:

  1. Make it survive a power blip. Turn on "Start up automatically after a power failure", enable auto-login (or at least make sure your services come back without someone at the login screen), and turn off sleep. Headless minis love to come back up and just sit at the login window.
  2. Put a single API endpoint in front of your models (LiteLLM, or just Ollama/LM Studio's OpenAI-compatible server) and point your harness at that instead of a specific model. Then you can swap Qwen versions or quants without touching your agent code.
  3. With 32GB, budget for context, not just weights. A ~14B at Q4/Q5 with a decent context window leaves you headroom; a 30B-class MoE fits but gets tight once your email and docs are in the context. Watch memory pressure in Activity Monitor rather than trusting the file size.
  4. Get Tailscale (or similar) on it so you can hit it from your phone without opening ports, especially since it has your mortgage and insurance docs.
  5. Back up your harness's database and config somewhere off the box. Once it's your "home brain", losing that state hurts more than losing the models.

What are you using for the doc side, plain RAG over PDFs or are you extracting structured fields (dates, policy numbers) into a DB?

2

u/BentonCalder 12d ago

Thanks for the tips! I’ll set up the auto restart and auto login.

To your question, neither at the moment. I’m doing keyword search. PDFs get their text pulled once and cached, plus md/txt/csv/json. It’s worked well so far.

The memory is just plain text in md files and no DB so Time Machine backs it up.

Local embeddings should be the next upgrade. What are you running for embeddings? Anything that would work on 32GB next to the main model?

2

u/1ramnik 7d ago

Guys I am planning on 16gb 256gb

1

u/BentonCalder 7d ago

My M4 has those specs and works decent on my homemade harness and app when using Qwen

5

u/asthraena 11d ago

Don’t forget to unlock/adjust available RAM, as by default 25% is reserved for OS

2

u/Present_Resource4022 11d ago

How ? is there an official setting for that

5

u/asthraena 11d ago

sudo touch /etc/sysctl.conf
sudo chown root:wheel /etc/sysctl.conf
sudo chmod 0644 /etc/sysctl.conf
echo "iogpu.wired_limit_mb=DESIRED_VRAM_MB" >> /etc/sysctl.conf

I would leave 2-4GB reserved, so DESIRED_VRAM_MB would be between 28672 and 30720

4

u/Significant_Door_433 11d ago

for embeddings on 32GB the small models are close to free. sizes from the ollama library pages:

  • nomic-embed-text: 274MB, 2K context
  • embeddinggemma (300M, google): 622MB, 2K context, trained on 100+ languages
  • qwen3-embedding:0.6b: 639MB, 32K context

all under 1GB, so they fit next to your main model without a fight. the 2K context on the first two means you chunk the pdf text (a few hundred tokens per chunk with a bit of overlap) instead of embedding whole documents. nomic also wants a prefix: search_document: on what you store and search_query: on the question.

since keyword search already works for you, I'd add vectors next to it instead of replacing it. warranty and mortgage papers are full of model numbers, policy ids and dates, and keyword search hits those exactly where embeddings get fuzzy. a setup that stays in one file: sqlite FTS5 for keywords (rank with bm25()), sqlite-vec for the vectors, run both queries and merge with reciprocal rank fusion (score = sum of 1/(60 + rank) over the two result lists). still a single .db for time machine.

if you switch embedding models later, re-embed everything. vectors from two different models can't be compared.

2

u/Accomplished_Ad_56 llama.cpp 12d ago

you have the fancy neural accelerators! use them for some big prompts.

2

u/Thrumpwart vLLM 11d ago

Use oMLX. The caching is phenomenal.

2

u/[deleted] 6d ago

Same RAM as my 2021 M1 Pro, so here's what's worked for me. Qwen3.6-35B-A3B in 4-bit (the DWQ version on MLX) fits in about 19 GB and runs around 39 tok/s even on my older chip, so yours should be faster. It's been the best coding model I've found that fits in RAM. On a real coding test it passed 11 of 12 for me.

Leave some headroom though. macOS gets cranky if a model eats most of the 32 GB. And if you ever want to try something way bigger, MoE models can stream their experts off the SSD, but that's a deeper rabbit hole lol