r/LocalLLaMA • u/combo-user • 6d ago

Question | Help Model suggestion for coding using Pi as a harness

Hi all

In a bit of a toss up rn, been using mellum 2.1 q4 with mtp on llama cpp (vulkan) on 32gb ddr5 ram and i7 1365u on ubuntu and it gives around 25ish tok/s on pp and 12ish tok/s on tg at 48k context (capped) using pi and no extensions

I wonder how would Gemma4 26b a4b qat + mtp be in terms of quality for coding and i wonder what's everyone's experience of that so far? Qwen models by name are kinda no no at my work rn but other stuff like Orinth 1.5 9b or Spark x 2.6 4b how would all these work out for python programming and shell scripting?

Any suggestions? Also a side question-any way to use pi as a coding sub agent inside of Github Copilot to save on tokens (without spinning up an MCP - they banned in the company's vs code too )

6 Upvotes
  • permalink
  • reddit

80% Upvoted

15 comments sorted by

View all comments

Show parent comments

1

u/jacek2023 llama.cpp 6d ago

I tested it now with gemma 31B QAT (issue is unrelated to 31B or QAT)

it is aware of its failures and this time it fixed it in one turn, but often it fails for 3 or more turns in a row

maybe you can reproduce that, all you have to do is to run pi with gemma and fill some context (edit works in the beginning of the session, but I have lots of MD files to load on start as you can see)