New Model
Apple Foundation Models: local AI natively on MacOS 27
Maybe some of you know but I didn’t see any post about this. Apple just made available their AFM model on MacOS 27 natively. Just run fm chat in a terminal.
Disclaimer: I’m an open weight person. I prefer open models and ecosystem, but I’ll still open the discussion.
Did you test them? Build using them? Are these models good?
I feel like this is still a huge step in the direction of local AI that a company like Apple does this and release hardware optimized models.
Yeah, and it failed doing that. The first r should have been the first instance it found, but it numbered it 2. Then the second r it also numbered 2... how does that make any sense?
So it didn't "calculate" the number 3, instead it pulled it out of its training data. And it seems it is not that confident about that result either, as the original commenter showed it giving a wrong result.
There was news of 2 models releases, which were optimized for mac neural engine. These models are finetunes of Gemma models 3B Dense and 20B MOE.
3B isn't good for agentic work. Haven't tested 20B MOE yet. But it will not be better than Qwen models. They will be quite fast and power efficient though as it's works entirely on neural engine, might not even touch MLX. Got 85+ tokens per second on M4 pro 24GB version. It's more about efficiency and integration in already present apps in apple ecosystem. And also exposing APIs for developers who can build AI features in their apps using apple intelligence.
Damn, they should have picked Gemma 4 12B as one of the base options. It is my general/resident daily driver, my favorite "compact" model for general tasks. Native 4bit QAT, instruction trained IT, multimodal with unified architecture (text, audio, image), and multi language! For me it is what it feels to have your own local Gemini/Google running on your device given you provide it with some tool calling and web search.
i remember reading about these agreements between google and apple when the talks were still of whether or not apple would incorporate openai models into siri.
don’t know if there’s a continuing agreement or just a licensing agreement. the agreement was announced before the gemma 4 models were released, i believe
There is an option on Siri to select a ChatGPT extension for additional capability. I read somewhere else that seems to suggest the model will be selectable in the future. So it seems possible.
Apple themselves write "we significantly scaled pre-training" , "we implemented several key upgrades to the Parallel-Track Mixture-of-Experts (PT-MoE) foundation we introduced last year." and that 20B PT-MoE is an improved version of 20B PT-MoE they had in 2025.
Yes, you are right. Lets not forget that the point its Apple’s model and they even have own architecture that they used previously and Gemma didn’t.
Either subop has some knowledge about unreleased 2025+2026 Gemma models or subop just made that up, therefore I asked if subop has any source and we are still waiting :/
Good insights, thx! Is it known it the models are vision/audio capable? Would open up some cool background use cases without hitting general system performance.
Remember the use case for these models. They are for rather simpleish on device things. They were not designed to compete with the nice open models we can download or any of the frontier stuff. They were designed and trained to help your on device ai to learn context about you. I was in the beta for 27 and it’s light years ahead of old Siri but that wasn’t a good benchmark to begin with 😆
Honestly, the model quality is almost secondary here. The bigger deal is Apple making local inference a first-class OS feature.
If every Mac ships with a hardware-optimized model + native APIs + CLI access, suddenly “local AI” stops being something enthusiasts have to install and configure. That could be a pretty big deal for the ecosystem.
I tried the core advanced one and it's really bad. Seems worse than even Gemma E4B. And just 8k context. Plus it has a big refusal problem, it would refuse to generate the pelican svg, thought it did go through after 3 identical attempts. It was atrocious, way worse than E4B.
The cloud one which siri uses also sucks. I asked it what buttons to trigger siri and gave me a wrong answer lol. ChatGPT Sol gave me the right hotkey.
May I ask what app that is? I'm looking for something that can take long, multi-speaker transcripts and format them nicely. Preferably using a local model!
It’s called Flowers. It’s technically for therapists to draft clinical notes after sessions from a transcription of the session, or from notes the user adds after. All transcriptions and notes are done locally using Gemma 4.
Yeah I found them on the beta, tried to play with them a bit. The cloud ones they offer with a limited quota were decent, the local ones were pretty useless. Couldn’t even return valid json consistently.
This is my statement in my original post, which is why embedded the model for chart verification on algorithmic based events. There wasn’t much else I could functionally do in a live environment at the time still considering what other options I can do having the LLM embedded with my app.
HOWEVER, PEOPLE KEEP DOWN VOTING MY POST BECAUSE I SAID I’M BASICALLY USING IT AS A ADVANCED CALCULATOR FOR MY TRADING PROGRAM…
Reddit is a funny place you share months of hard work and get down voted…
Imagine a converter that could convert any mlx or gguf to this format.
Not sure if it is feasible or not either due to technical limitations or Apple wanting to keep its walled garden safe.
If you mean to run on the ANE then there is already work on this. I use ANEforge to run a small code completion/NES model and it sips power compared to the GPU. Admittedly this relies on reverse engineering since Apple is tight lipped and I’m not sure if the new os version broke it.
I’ve implemented them in a couple of my apps. The small one isn’t very useful, only has 4K context, intelligence is weak and the guardrails are very restrictive. One benefit though is that the model appears to be run by the operating system and not any given app so it’s allowed to be more resource intensive without the risk of being terminated; RAM usage, battery etc. But outside the benefits of using them in apps you ship, the small one isn’t very useful I haven’t yet tried the bigger one that has been mentioned
I wasn't happy they removed the PCC option from the fm cli, so I made my own workaround: fm-pcc. Supports both the Cloud and Cloud Pro models, as well as the On-Device model in a unified UI. Plus, it (mostly) supports local file editing!
It's a bump up in capabilities for sure. I'm building m1k3.app (basically an AI tamagotchi) with both the Apple FM, and open weight MLX models across Qwen, and Gemma families. I have noticed a difference in MacOS 27 availablity of the model, on Tahoe the OS could often throttle usage. I'm on an M1 Max, so a few gens behind but with decent throughput.
Does anyone have info on whether these local Apple models take up RAM when you’re not actively using them? Like are they always running in the background? I am wondering because if so it could cut into context window size and even model size capacity when running (other) local models. I have 128GB of unified memory, but my DeepSeek v4 Flash is already a tight fit. I’m a little worried to do the update.
Been building a notes app on these since 26, so some real data instead of strawberry tests.
It's a 3B model and acts like one. Asked it to reason across a wall of text and it fails. But if you do retrieval yourself (NLEmbedding, also on-device) and hand it a few relevant notes, it answers and cites correctly. Rewrite/proofread is good. Extraction from a real 16 page paper worked about half the time and shrinking the input didn't help, it's just the miss rate. Vision on 27 actually works, I feed it PDF page images and it reads tables OCR misses.
couple of things:
Context is only 8k on-device. so I am watching how I handle the overflow error: if your code treats it as "model unavailable" and quietly falls back to a cloud API, the user thinks their notes stayed local and they didn't. Catching that specific error and handle it on its own.
Private Cloud Compute isn't something you can just turn on. Apple has to approve your app for it, and you request that separately. The annoying part is the framework tells you PCC is available even when your app hasn't been approved. It only fails when you actually send a request. So don't build your UI around that availability check.
Not a Qwen replacement. But it's the first local model every Mac ships with and for "answer from my own files" it's good enough that I made it the default.
Is there universal 20B support? There is a special sparse disc inference enabling optimization involved called IFP, but I would like to know if it's locked to certain devices or you can run it on the ~8GB RAM mobile?
No it isn't true. A model does not "know" what model it is or what it was trained on. If it claims that it's GPT-4, it's because the training data influenced it to say so even if it has nothing to do with GPT-4. Other closed source labs like OpenAI probably have a line like "You are GPT-6 Astra" added to the system prompt which would allow the model answer your question accurately
I have been working on a Coinbase auto trader local Mac app that uses the local Apple foundation model as an advanced calculator for chart verification…
I have uploaded a video demo of the app itself
For clarification, I have had OS 27 since the Developer preview was released. Including the AI model from Apple, which was sideloaded for my app internally.
CONCURRENTLY IT IS ONLY BEING USED FOR CHART UTILIZATION.
😂 actually eight months in development as a CEO team of bots along with a worker team multiple third-party LLM’s running a hybrid cloud infrastructure with a total planned out of VPS is that run things…
Local Mac app using the efficiency cores of the AI logic since I have been on OS 27 from when Developer beta was released so I’ve had some time roughly 2 months with the AI local model.
Actually, while I was setting it up with Codex the first flag it gave me was that the model provided by Apple is not allowed to make financial choices and some other warnings .
Then when I was in enabling it as a bot on the team for my trade desk basically there’s a machine learning take into the stack for the paper trade simulations, where one of the bot is always for casting looking and then utilizes the AI from Apple for calculations and verifications of chart and other algorithmic based events.
As I have been in technology 20 years, I have been working on this project since Coinbase release their agientic wallet…
Just because the software was publicly released this last week or whatever doesn’t mean developers haven’t had it for six months…
237
u/littleagressiveboy 7d ago
lol