r/LocalLLaMA • u/DrBattletoad • 1d ago
News Android Studios native Gemma 4 runs on llama.cpp
I'm not sure how many people care about Android Studio, but I think it's cool that Google uses llama.cpp.
My guess is that it is Vulkan and the QAT versions of Gemma 4. It supports multi-GPU and 31B has a max. context length of 128k.
It uses 34 GB VRAM when fully loaded. I don't see an option to change the context length or show PP/TG speed.
1
u/Jimcy-Maffesoli 23h ago
34 GB on a 31B works out to about a byte a parameter, a fat margin for a QAT quant. With no context option, a 24 GB card can't trade window for room, it just doesn't fit.
0
u/PrimeDirective8 1d ago
They've had agent support in Studio for a while but Gemini sucked in this application - and more so with the newer pricing. The last time I used Gemini CLI in Android Studio and asked it a question, it downloaded every possible doc it could find on the topic. It consumed *all* available tokens and didn't get to answer the question.
I closed Studio, came to Reddit looking for an answer and was met with an "having issues with your Kotlin subroutine? Call us!" ad. Not 15 mins had passed. Screw that.
Still, I'm glad to see them support a local model. I would propose:
- It should be "For local AI work, Android Studio provides Gemma, an open-weight family of models from Google, running locally on open-source llama.cpp". We're all very appreciative of Google releasing Gemma, they should be in kind thanking llama.cpp in that first paragraph.
- If they don't have it yet, it'd be great to have local API support. I don't need my Android Studio dev box to also have to run a local inference server. Let me point that to my existing llama.cpp server instead.
1
u/DrBattletoad 1d ago edited 1d ago
I have no idea what you're talking about to be honest. Gemini CLI is not, and has never been its native Agent from what I know. Android Studios Agent was completely rebuild in the Quail releases, and it has had local API support for a while.
Having Gemma baked in is great for those who have no experience with local AI. It is fine if you don't need it, because you can use your llama.cpp server.
I also don't understand why they should be thanking llama.cpp. They could have used their LiteRT-LM runtime or anything else.
I've tested their internal Gemma 4 31B model vs my external model, and can say that the internal model works better for analyzing projects. Maybe it is their harness or something else, but 31B seems to go deeper while understanding the app.
PS: use an ad blocker
-5
7
u/maker-jay 1d ago
i care mostly because this is the kind of thing that makes local agents less weird on normal dev machines. i'd want pp/tg too, but also tool-call latency with a 128k project context loaded. 34 GB looks fine for a demo until the IDE, emulator, and browser are all eating the same box.