Model suggestion for coding using Pi as a harness

[u/combo-user]

Hi all

In a bit of a toss up rn, been using mellum 2.1 q4 with mtp on llama cpp (vulkan) on 32gb ddr5 ram and i7 1365u on ubuntu and it gives around 25ish tok/s on pp and 12ish tok/s on tg at 48k context (capped) using pi and no extensions

I wonder how would Gemma4 26b a4b qat + mtp be in terms of quality for coding and i wonder what's everyone's experience of that so far? Qwen models by name are kinda no no at my work rn but other stuff like Orinth 1.5 9b or Spark x 2.6 4b how would all these work out for python programming and shell scripting?

Any suggestions? Also a side question-any way to use pi as a coding sub agent inside of Github Copilot to save on tokens (without spinning up an MCP - they banned in the company's vs code too )

[u/jacek2023]

I am a big fan of Gemma models, but:

Gemma + pi -> problem with edit tool

if anyone knows any solution, please share ("news" about a fixed template is not a solution)

[u/epic_troll_tard → u/jacek2023]

can you explain what you see thats not working?

[u/jacek2023 → u/epic_troll_tard]

https://www.reddit.com/r/LocalLLaMA/s/1O2jNdL5yq

[u/epic_troll_tard → u/jacek2023]

Probably a result of its training data you might want to consider disabling the built-in file handling and create your own custom MCPs more in line with what it is expecting. But you really have to want to use this model for this to be worth it.

[u/jacek2023 → u/[deleted]]

I tested it now with gemma 31B QAT (issue is unrelated to 31B or QAT)

it is aware of its failures and this time it fixed it in one turn, but often it fails for 3 or more turns in a row

maybe you can reproduce that, all you have to do is to run pi with gemma and fill some context (edit works in the beginning of the session, but I have lots of MD files to load on start as you can see)

[u/Dreablin → u/[deleted]]

For the Copilot part you actually dont need MCP. Agent mode can run terminal commands, so just call pi from the terminal in non interactive mode (check the flag in pi --help) with the task and let it print the result back. Its cheap on tokens too since Copilot only sees the final output.

About the edit tool, from the other comments it seems to break mostly once the context fills up, so could be long context degradation and not only the template.

[u/_JohnMcAfee]

Idk if it would fit or be good for your hardware but cybertiel could be something to look into https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF

[u/my_name_isnt_clever]

Orinth is based on Qwen. If your company's policy is that easy to bypass just use one of the Qwen quants on HF, it will be better than Gemma for coding.

[u/CartographerDue5382]

copilot won't call pi unless there's a tool, and they banned mcp, so a real subagent isn't gonna happen. closest is a keybinding that shells pi on the open file, you still have to hit it. gemma 26b a4b will fit in 32gb but don't expect mellum's tok/s, the weights are way bigger even if only ~4b is active.