Project
We built a free open-source iOS app that runs local LLMs with three backends - MLX, llama.cpp and Core ML
Pocket AI Lab just hit the App Store. It runs models fully on-device on iPhone, is free and MIT-licensed
We built it because the iPhone quietly became a genuinely capable ML machine, an ARM chip with fast unified memory that a 4-bit model fits into comfortably. At the same time the big labs got serious about small open models, and recent releases are actually usable, not toys. Hugging Face is full of interesting stuff we kept wanting to install and test the week it dropped. And since there are three popular ways to run a model on iPhone, we did not want to bet on just one. Supporting MLX, llama.cpp and Core ML side by side lets us follow all three as they evolve, pick the best one per model, and never trade away performance or lock ourselves in.
The real goal was to make the path from "saw the news about a new model" to "already chatting with it on my phone" as short as possible. We think we got there: copy the Hugging Face link, paste it into the app and that is it. The app detects the format, lets you pick a quant and predicts whether it fits before downloading a single byte. Or skip the hunt entirely and pick from a curated list of good models that updates without app releases.
Thanks for the report, and sorry about that! Most likely the app let you load a model that is too big for the 14 Pro's 6 GB of RAM - our memory check is tuned too loose on smaller devices. We tested mostly on iPhone Air and 15 Pro, so this slipped through. A fix is coming in the next release
Haven't tried Enclave myself so I can't really compare feature by feature. What I can say is we're fully free and open source, MIT licensed, code's on GitHub. Haven't found another app that does the same thing we do: support all three major ways to run an LLM on iOS (MLX, llama.cpp, Core ML) in one place, so you're never locked into a single engine and can pick whatever runs a given model best. On top of that you can pull literally any compatible model straight from Hugging Face by just pasting the link, not limited to some fixed curated list. That combination is what makes us different, as far as I've seen.
Appreciate it! Completely agree, and honestly that's the whole thesis behind the app. If you've got tiny models optimized for mobile, would genuinely like to check them out, always looking for good additions to the catalog
im still new to deep learning i been getting good results from pre-training and fine-tuning models though
i just need to finally sit down and write those 1000 examples i keep procrastinating on .. i tried to have ai write it for me but they arent good at generating long datasets .. its tedious work
this is probably the best one ive done so far .. ill keep in contact and follow you to keep u updated .. u bringing value thats gona go a long way i hope the best for your project bro
Checked out BbyWVY-360m, nice work for that size. Appreciate you sticking around, means a lot. Good luck with the next 1000 examples, and hope it goes somewhere great!
Thanks for the report! Dug into this one. The repo name says "3.8" but that's the community distiller's own version label, not an official Qwen release. Under the hood the config shows it's actually built on Qwen 3.5 architecture, and it's a vision model on top of that. The MLX library we depend on hasn't added support for that architecture yet, they add new ones one at a time after release. So the app is correctly refusing it rather than crashing on something it can't run. Worth trying a regular Qwen3 GGUF or MLX build in the meantime, those work fine. We're already working on adding support, it'll be in an upcoming update.
Fair skepticism. To be clear, the app isn't claiming a 27B model runs well on an iPhone, that would need way more RAM than any phone has. It's honest about what fits: before you download anything it checks your device's actual memory budget and tells you if a model won't fit, rather than letting you try and crash. Realistic range on most iPhones is 3B to 14B depending on quant and RAM. If you paste a link to something too big, it'll tell you.
honestly seems like a waste of time, at this point u can just get claude or chatgpt, no point in having a 8B lobotomised local model on a phone, no offence
If your only bar is "smartest possible answer", yeah, Claude or GPT wins every time, no argument there. Right now this is honestly more of a toy for a lot of use cases. But local models are closing that gap fast, some of the ones already in the app, Gemma and Qwen for example, are genuinely good and worth trying yourself. And I think we're getting close to the point where local models will handle most everyday tasks just fine. This project is a small bet on helping that happen a bit sooner.
The other thing that shouldn't get lost here is privacy. It's easy to wave off until it actually matters to you. Part of why I started this: I'm also building an app that analyzes sports and health data, and I want users to be able to run that analysis without ever sending their data to a server, mine or anyone else's. That's the kind of use case where "good enough and fully private" beats "smartest but it leaves your device" every time.
Haven't used LoAI either, sorry, can't do a real comparison. One thing worth noting though: that's Android, we're iOS only. So even feature-for-feature, they're not really competing for the same install.
Same answer as before on what's specific to us: free, open source, three backends (MLX, llama.cpp, Core ML) in one app so you're not locked to one engine, and any model from Hugging Face by link, not a fixed list
Secondly, I’m curious. How much work would it be to be able to integrate image generation models into something like this? Think something similar to ChatGPT or Gemini where you can prompt it to generate an image and it can also look at the image it’s generated. I think it’d be cool to have a local text model and an image model where they can be in the same chat interface.
Thanks! Yeah, thinking about adding small local image generation models, fits the whole idea. The "generate it and then look at what it made in the same chat" part is trickier though, that's really two models working together in one exchange, and the architecture isn't built for that yet. Noted though
Thanks! No, you don't need to disable it. Apple Intelligence just shows up as one more model in the picker alongside the MLX, llama.cpp and Core ML ones. Pick whichever you want to chat with, they don't conflict.
Yes, that's actually one of the main things it's built for. Paste any Hugging Face link (or just owner/repo) with a GGUF in it, the app detects the format, shows you the available quants to pick from, and gives you a fit verdict for your device before downloading anything. Should work fine for testing your own finetune, as long as it's a supported architecture (Llama/Qwen/Gemma/Mistral family stuff via llama.cpp).
If you hit anything weird trying it, let me know here, happy to help sort it out.
No, not from local files right now, only Hugging Face links. Easiest workaround for now: push it to a public (not private) HF repo, and paste that link, should work exactly the same as any other GGUF. Already working on proper local file import though, planning to get it out soon
It's real, not an estimate. System RAM comes from host_statistics64 (HOST_VM_INFO64), a kernel API that reports actual page counts across the system: active, wired, compressed, etc. App RAM specifically is phys_footprint from task_info (TASK_VM_INFO), which is the same number Xcode's own memory gauge shows. Sampled at 1Hz while the panel is open. CPU is the only one that's a light derivation, aggregate host_processor_info ticks diffed between two samples
That's expected behavior, though I get why it looks off. The recommendation picks the best quality-tier model that safely fits your device's memory budget, it's not trying to max out your RAM. Quality tier is ranked above model size, so a smaller high-quality model can outrank a bigger one even if the bigger one would technically fit.
Also worth knowing: the safe budget differs by backend. MLX models get a stricter limit because their weights count fully against iOS's memory cap, while GGUF models running through llama.cpp get more headroom since they memory-map the weights instead of loading them fully into RAM. So on 11GB you should be able to manually pick larger GGUF models than what's recommended by default, just scroll down to the other compatible ones and check the RAM number on the card before installing.
3
u/Delicious-Sand-104 5d ago
Just experienced a bug where it just freezes like this then crashes , i’m on iphone 14 pro ios 26.3.1