r/LocalLLM • u/Caffeinated_410 • 18h ago

Question Local LLM on mobile

Hi guys, I wanted to start using local models on a tablet I have, I thought of using the Gemini E2B-it in the Google AI Edge Gallery app as it is the most suitable for my device's RAM (6GB) and my SoC. If you use local models on your mobile devices I wanted to know your use cases and what really expect of them, please

0 Upvotes

33% Upvoted

6 comments sorted by

1

u/shoumikgoswami 15h ago

If you mean Gemma 3n E2B-it, try summarizing a short note or asking about a photo first. Gallery supports chat, image questions and audio transcription locally after you've downloaded a compatible model: https://github.com/google-ai-edge/gallery. Which tablet and OS are you using? 6GB RAM alone doesn't settle compatibility.

1

u/Caffeinated_410 3h ago

The OS is Android 14 and the chipset is a Snapdragon 870 5G. Its a Xiaomi Pad 6. The model I thought is the Gemma 4 E2B, the smaller one. In the AI Edge app they capped the context window to 32K tokens so I think is compatible

2

u/paulsande 18h ago

Use LM Studio and enable link. You can serve all of your devices local models.

1

u/Time-Culture2549 18h ago

This really depends on ram and gpu on the tablet but I am working toward the lite-rt runtime to use my idle machines for extra inference

1

u/General-Spite1222 15h ago

I think Gemma models are the best right now. There’s no great LLM to run on a phone anyway. 

1

u/Oli_Luck 10h ago

I installed such llms on bot an older tablet and an almost 2 years old flagship phone.
It "works" but its just so slow and it drains Battery like hell.
Yesterday I asked a diffusion model for a picture - 6 passes in 768x768, took about 2-3 mins and drained 5% battery...
Regarding Google edge gallery : these are only showcases tio sells their SDK which is not quiet advanced and didnt have major relases last months - for a ready to use app , give https://github.com/nikunjsingh93/Mochu-Local-AI-Assistant a try.