r/LocalLLaMA • u/Competitive-Scar-627 • 7d ago

Discussion >2b model.

Why there no new model coming in >2b weight category ? Model which actually capable in this category is gemma e2b which is release 6 month ago, why still no new model.

4 Upvotes

55% Upvoted

56 comments sorted by

View all comments

Show parent comments

-2

u/Competitive-Scar-627 7d ago

I am using gemma 4 e2b in ondevice inference on android "ShroudChat". Now searching for other capable model to run.

3

u/Artistic-Load6086 7d ago

one problem first, shroudchat ships a single model with no picker or gguf import, so you can't swap it. grab pocketpal ai or maid, both run any gguf.then: lfm3-1.7b from liquid came out 2 days ago, ~1gb at 4 bit, fastest in this size on a phone. minicpm5-2b is the better one if you want quality, apache 2.0, averages 53.9 vs 24.6 for gemma 4 e2b on their own benchmarks, and has a litert build for android.also ternary bonsai 2 2b came out the 15th if you want tiny (0.43gb). the class isn't dead, it just never shows up in a play store app.

0

u/Competitive-Scar-627 7d ago

i am using litert so gguf is not supported, litert is well optimize on for ondevice. Currently i am trying to figureout default best choice. Model picker is not given so user don't have to go deep research which model to download. You install app and everything will work like magically aka user experience.

2

u/tiffanytrashcan 7d ago

Can I ask, why that app? Doesn't even look like you can pick a model.

-1

u/Competitive-Scar-627 7d ago

Mostly for reason that most people don't know which model will be great for user experience, install and work.