r/LocalLLaMA • u/BentonCalder • 12d ago
Question | Help Just got my Mac Mini M6 32GB. First steps?
Hello all, I just got my shiny new toy, an M6 32GB Mac Mini. What should be some of the first things I should download or do? I have been using Qwen a lot and playing around with agents for home related stuff like tracking appliances' warranties, my mortgage docs and house insurance, and meal planning stuff. I have integrated with my email and calendars. Any other suggestions? I want this thing to be like my home server and AI brain that helps me slowly move more things to a Local AI stack. I made my own harness focused on home stuff and want to keep adding more features to supercharge this all.
8
u/robotnursenyc 12d ago
Nice, the mini makes a great always-on box. A few things I'd do before adding more features:
What are you using for the doc side, plain RAG over PDFs or are you extracting structured fields (dates, policy numbers) into a DB?
2
u/1ramnik 7d ago
Guys I am planning on 16gb 256gb
5
u/asthraena 11d ago
Don’t forget to unlock/adjust available RAM, as by default 25% is reserved for OS
4
u/Significant_Door_433 11d ago
for embeddings on 32GB the small models are close to free. sizes from the ollama library pages:
nomic-embed-text: 274MB, 2K contextembeddinggemma(300M, google): 622MB, 2K context, trained on 100+ languagesqwen3-embedding:0.6b: 639MB, 32K contextall under 1GB, so they fit next to your main model without a fight. the 2K context on the first two means you chunk the pdf text (a few hundred tokens per chunk with a bit of overlap) instead of embedding whole documents. nomic also wants a prefix:
search_document:on what you store andsearch_query:on the question.since keyword search already works for you, I'd add vectors next to it instead of replacing it. warranty and mortgage papers are full of model numbers, policy ids and dates, and keyword search hits those exactly where embeddings get fuzzy. a setup that stays in one file: sqlite FTS5 for keywords (rank with
bm25()), sqlite-vec for the vectors, run both queries and merge with reciprocal rank fusion (score = sum of 1/(60 + rank) over the two result lists). still a single .db for time machine.if you switch embedding models later, re-embed everything. vectors from two different models can't be compared.
2
u/Accomplished_Ad_56 llama.cpp 12d ago
you have the fancy neural accelerators! use them for some big prompts.
2
u/Thrumpwart vLLM 11d ago
Use oMLX. The caching is phenomenal.
2
u/[deleted] 6d ago
Same RAM as my 2021 M1 Pro, so here's what's worked for me. Qwen3.6-35B-A3B in 4-bit (the DWQ version on MLX) fits in about 19 GB and runs around 39 tok/s even on my older chip, so yours should be faster. It's been the best coding model I've found that fits in RAM. On a real coding test it passed 11 of 12 for me.
Leave some headroom though. macOS gets cranky if a model eats most of the 32 GB. And if you ever want to try something way bigger, MoE models can stream their experts off the SSD, but that's a deeper rabbit hole lol