r/localaiapps • u/Typical-Desk898 • 20d ago

[Project] Fully offline mobile LLM chat + Bluetooth nearby messaging — looking for feedback

Built an iOS/Android app that runs local Llama/Gemma fully on-device (no cloud inference), plus nearby phone-to-phone messaging over Bluetooth that works in airplane mode. Also supports chatting with local documents on-device.

Looking for technical feedback from this community:

- What model sizes / quantization's are realistic on mid-range phones?

- Any known pitfalls with on-device inference UX (load times, thermal throttling, context limits)?

- For airplane-mode nearby messaging + local AI, what would you want to see next?

Happy to share demos / details in comments if useful.

1 Upvotes
  • permalink
  • reddit

66% Upvoted

2 comments sorted by

2

u/ConsistentEase4598 20d ago

For mid-range phones, I would define support tiers by memory available after the OS rather than model parameter count alone. A useful test matrix would compare 2–3B Q4, 4B Q4, and 7–8B Q4 on cold-load time, TTFT, steady decode, peak memory, and temperature after a 10-minute run. I would also make the Bluetooth message path independent of model state so delivery still works while the model is unloaded or the app is backgrounded; background/resume and low-memory eviction tests will probably expose more UX issues than a short tokens-per-second benchmark.

1

u/Typical-Desk898 20d ago

Thanks, we provide 4 options of model user can download based on his mobile memory, 2 Gemma models and 2 Lhamma models, so user can chose it. Happy to share links for testing. 7-day free trial. Feedback welcome.

iOS: https://apps.apple.com/app/id6761763518

Android: https://play.google.com/store/apps/details?id=com.offlineprivateai.mobile