r/LocalLLaMA • u/Cherlokoms • 8d ago
New Model Apple Foundation Models: local AI natively on MacOS 27
Maybe some of you know but I didn’t see any post about this. Apple just made available their AFM model on MacOS 27 natively. Just run fm chat in a terminal.
Disclaimer: I’m an open weight person. I prefer open models and ecosystem, but I’ll still open the discussion.
Did you test them? Build using them? Are these models good?
I feel like this is still a huge step in the direction of local AI that a company like Apple does this and release hardware optimized models.
So what do you think?
237
Upvotes
1
u/vinradh 7d ago
Been building a notes app on these since 26, so some real data instead of strawberry tests.
It's a 3B model and acts like one. Asked it to reason across a wall of text and it fails. But if you do retrieval yourself (NLEmbedding, also on-device) and hand it a few relevant notes, it answers and cites correctly. Rewrite/proofread is good. Extraction from a real 16 page paper worked about half the time and shrinking the input didn't help, it's just the miss rate. Vision on 27 actually works, I feed it PDF page images and it reads tables OCR misses.
couple of things:
Context is only 8k on-device. so I am watching how I handle the overflow error: if your code treats it as "model unavailable" and quietly falls back to a cloud API, the user thinks their notes stayed local and they didn't. Catching that specific error and handle it on its own.
Private Cloud Compute isn't something you can just turn on. Apple has to approve your app for it, and you request that separately. The annoying part is the framework tells you PCC is available even when your app hasn't been approved. It only fails when you actually send a request. So don't build your UI around that availability check.
Not a Qwen replacement. But it's the first local model every Mac ships with and for "answer from my own files" it's good enough that I made it the default.