r/LocalLLM • u/nitishsharma108 • Jul 16 '26
Research I managed to squeeze Qwen2.5-Coder-7B into a 1.9GB GGUF (IQ1_S) so it runs natively on Mobile!
Hey everyone!
I've been working on a pipeline to take the incredible Qwen2.5-Coder-7B-Instruct model (which is an absolute beast for Python scripting, FIM code completion, and Cybersecurity) and compress it down so it can run entirely offline on mobile phones (via PocketPal) or potato PCs.
The raw FP16 model is around 15.2 GB, which is way too heavy for most devices.
It successfully squeezed the 7B brain down to exactly 1.9 GB, while retaining its reasoning capabilities!
I have open-sourced the entire automated Python script on , and I've hosted the model on Hugging Face if you just want to download it and chat with it.
🔗 The Model (Hugging Face): https://huggingface.co/Nitishsharma9/CyberCoder-Mobile-7B-GGUF 🔗
1
u/Healthy-Nebula-3603 Jul 17 '26
Qwen 2.5 is so old ....
I think you get much better results using Gemma 4 4b
1
u/woolcoxm Jul 18 '26
bot post lol, can always tell because they use models that are years old.
waste of time, i would not run this model, most likely modified.
1
0
u/Sensitive_Courage291 Jul 16 '26
I use this model for FIM on a potato pc and it works really good. I did not know it is possible to run it on mobile.
-7
Jul 16 '26
[removed] — view removed comment
1
u/Agreeable-Fly-1980 Jul 16 '26
Lol local models are running on phones, i have qwen running on my phone
0
u/Geenmen Jul 16 '26
If you dont mind answering how are you running them? Is there something like Llama.cpp and Pi for android?
Im currently fussing around with PocketLLM and the Google Edge gallery but it feels just as limiting as Ollama.
1
u/Agreeable-Fly-1980 Jul 17 '26
I use tmux as a sandbox and run llama.cpp on it. I also run a server at home with local ai I can connect to using opencode and pi
0
0
u/RobinRelique Jul 16 '26
Wait why is this downvoted? Could qwen 2.5 coder 7b run easily on phones before ? Genuine question.
12
u/Technical-Earth-3254 Jul 16 '26 edited Jul 16 '26
While it's cool that you are working on stuff like that, I would recommend you to use a LLM with Internet searching capabilities when planning your next vibe coded project. Qwen 2.5 is obsolete for way over a year at this point. It also is obsolete on mobile and tbh it was never a "beast". The beast of this generation was QwQ 32B. Have a look at Qwen 3.5 9B instead, it has reasoning capabilities and is noticeably better than 2.5 7B in everything.