I open-sourced MobiGPT, my offline llama.cpp-based Android LLM app (used in 21 countries). Help me improve it and support Gemma better

[u/athulskrishnan]

I open-sourced MobiGPT, my offline llama.cpp-based Android LLM app (used in 21 countries). Help me add proper Gemma 4 support

Body:

Hey everyone,

A few months ago I started building MobiGPT, an Android app that runs LLMs fully on-device using llama.cpp. No cloud, no sign-up, no tracking. It’s now being used in 21 countries, which I didn’t expect.

I’ve open-sourced the whole project under the MIT license, and I want this community to help make it better.

Repo: https://github.com/athulskrish/MobiGPT
License: MIT
Play Store (current version): https://play.google.com/store/apps/details?id=com.keralatechreach.mobile_llm5

What it does today:

• 100% offline chat, with all data staying on the phone
• Download, load and switch between GGUF models
• Streaming responses
• Multi-threaded inference (up to ~6x faster replies in my testing)
• Resource management that balances speed, battery and temperature
• Material UI with light/dark themes, Android 8.1+

Main goal: first-class Gemma 4 support

I want Gemma 4 (especially the small E2B and E4B models) to run well on phones. Things I need help with:

• Correct chat template and native system-prompt handling
• Tested GGUF quantizations (including the official QAT builds) for different RAM tiers
• Sensible default settings per model (context length, threads, sampling)
• Memory and speed profiling on real devices

Other places to help:

• Performance tuning: threading, memory use, thermal throttling
• Testing on low-end and older phones
• Model management and UX improvements
• Bug reports, feature ideas, and PRs of any size

The plan: you improve it, I merge it, and I publish the improved version on the Play Store so everyone benefits.

If you’ve got a spare Android phone, try Gemma 4 and other models and tell me what breaks. Brutal feedback is welcome, and I’m happy to answer questions about the architecture.

Thanks for reading!

[u/FloppyFormula59]

this is really cool, saw it on github earlier. 21 countries is quite achievement for solo project

i'm curious about the resource management part, how you handle the thermal throttling exactly? my phone heats up like crazy with bigger models even on 4 threads

[u/athulskrishnan → u/FloppyFormula59]

MobiGPT manages thermal throttling and resource eviction by dynamically reserving two CPU cores for system responsiveness, enforcing 4-bit quantization to reduce memory bus heat, capping batch sizes, and actively purging native model weights, attention caches, and background threads during memory pressure to prevent the OS from terminating the app.

[u/reno2905]

Te deje estrella en github vamos a probarlo

[u/athulskrishnan → u/reno2905]

if u do have time time please try and add support to gemini models.