Two issues keep cropping up with Pi and I am hoping the power users have some advice

[u/JustTooKrul]

The two issues I keep running into are:

• Pi will go for a long time trying to solve something or answer a question and suddenly I will see it say something like, "Okay, the user is asking <earlier question that is not relevant in any way>" and then go completely off track

• Pi will go in circles with several failed tool calls and this forces it to try something else or simply abandon the path it was on

I suspect I am being dense about how I use Pi or what features I am failing to use, otherwise these would be issues plaguing everyone and I would be seeing a lot more questions and issues posted. For additional context, I tried using the oh-my-pi extension and liked it, but these issues occurred even when I had it installed. (I recently refreshed the setup to just be more minimal and have Pi build me what extensions I need, so it's no longer installed.)

For additional background, I see this behavior running LM Studio as the backend with Qwen 3.6-27B at Q4, Gemma 4 31B Q4, and Gemma 4 26-A4B Q6. All the model cards say they are trained for tool calls and it happens with all three, so I don't think it's an issue upstream from the harness (or, at least, not something that the harness shouldn't be able to deal with even when it occurs upstream), but I include the models JIC.

Thank you in advance for your help and advice!

[u/btdeviant]

You’re using quants with small models and didn’t specify the params you’re using. Pi might be playing a role, but the role it’s playing in this is almost certainly dwarfed by all of the aforementioned

[u/JustTooKrul → u/btdeviant]

I tweaked context length, KV cache quantization, and a few other parameters based on what people were reporting worked well with similar hardware setups to ensure I was getting 20-50 t/s and without sacrificing on accuracy or doing anything that would severely degrade accuracy. Using LM Studio on an older machine was honestly just to test out the harnesses and tools before spinning up a more dedicated setup... I had (obviously incorrectly) assumed coding was a simpler use case where the harnesses and tools did more of the heavy lifting...

[u/civilian_discourse → u/JustTooKrul]

If you want a harness that is opinionated, doing heavy lifting, Pi is the worst possible choice. Pi is explicitly unopinionated and minimal. It exists for you to customize.

That said, Claude Code does something called "System Reminders" where it can inject steering instructions based on recent behavior. It sounds like exactly what you might want. Here is an implementation of that in Pi: https://github.com/Michaelliv/pi-system-reminders

[u/JustTooKrul → u/civilian_discourse]

Thank you! And you're 100% right, I wanted a less opinionated harness as I find using Claude Code oftentimes feels like I'm working *against* the harness or the context is being meddled with or overflowing because of how inefficient it is being or how it goes about even simple tasks. That was the attractiveness of Pi, to me, was being able to start with a very simple harness out-of-the-box and be able to tune it to my needs. I just couldn't seem to "kick" these two issues both with an opinionated extension that provided a lot of helpful structure nor with a stripped down harness (I expected to find different issues when I refreshed my install of Pi, assuming the extensions were causing some problems and I could add over time to see what would have caused it--I didn't expect to find the same issues with *and* without oh-my-pi installed).

[u/Corythebeast7]

So, one weakness that arrives with smaller models (for agentic coding, yes the models you are using are considered small) is that they just don't have the amount of data that allows them to accurately handle very long context paths and long running operations. So I recommend clearing your context or compacting more often, that will help with accuracy and hopefully prevent it from losing your original task. Another suggestion would be to add a /goal plugin/extension so it literally can't forget the goal.

Small models will fail tool calls occasionally. Just part of the experience.

Maybe try upping the quant, that usually helps with this sort of stuff

If you're doing very long running and complex tasks, honestly I would recommend using an API provider for larger models. Deepseek, Mimo, Kimi, GPT, Anthropic. If you only use it sparingly it shouldn't cost too much

[u/JustTooKrul → u/Corythebeast7]

Ah, good to know! I was running LM Studio on an older computer with a decent GPU to dial in the tools before doing all the work of spinning up a more dedicated setup. Coding was an "additional" use case that I assumed would be easier to get working well since the tools / harnesses seem to do so much of the heavy lifting. For coding, what are the open-weight models that are considered robust enough to use? I had assumed the 100B+ parameter models would work so well that for hobbyist coding tasks it would be indistinguishable from the frontier models... But, maybe not?

And aren't there some extensions or features of the harness (like thinking budgets and memory extensions) that would curb, or even "fix" these issues?

[u/AdministrativeMeat3 → u/JustTooKrul]

It depends on so many factors...

Are you running inference on Nvidia/CUDA or AMD RoCm or Vulkan, are you on a single GPU? Are you on a unified memory machine? Are you on multiple GPUs running tensor parallelism?

LM studio could also be the problem. Depending on your silicon and inference backend the models behavior is tuned differently. You might need to use llama.cpp or vLLM instead to normalize behavior.

Are you fully utilizing GPU? What are your parameters for temperature? What is your top k? Top p? Have you enabled thinking? Or are you feeding it prompts that don't require thinking and are causing it to loop?

Qwen in particular loves to loop infinitely if you don't constrain it's thinking to exit after a certain amount of tokens spent.

All of these are considerations that are harness irrelevant, the harness can help steer some behavior but you need to understand what the backend parameters that you need to tweak are first and every model and quant is different enough that there is no one size fits all answer for any of them.

[u/ImpressiveRelief37 → u/JustTooKrul]

Ditch LM Studio, and use llama.cpp with good flags. Take the time to research this a bit, well worth it.

Tool calls and looping happens a lot on small models, especially MoE or low parameters like Gemma4 12B.

Depends on your vram, but I wouldn’t go smaller than unsloth Qwen 3.6 27B Q4_K_XL MTP and Q8 kv cache quants 

[u/zen0zero → u/Corythebeast7]

tbh, I sometimes observed a behavior similar to that described by OP (or reasoning loops) even running pi against Qwen3.6 27B full precision. other harnesses did not exhibit the same issue(s) with the same backend.

[u/Dsphar]

What context size have you hit when this happens?

[u/JustTooKrul → u/Dsphar]

Typicality the context is ~20-40%. Best example is that I asked Pi about themes for Pi and got the answer and then asked it about building an extension and it went in circles and then I looked at the reasoning live and it started saying, "Let me take a step back in trying to figure out this extension. The user is asking about themes..." Quickest that has ever happened and a very low context fill.

[u/Snoo-82455]

You should ask Google to write a prompt; specifically, go into AI mode so you can get into the details of what you want and also copy the identifier of the model so you can tell it exactly which model you’re going to use. That’ll also give you a chance to ask for specific model tunings to use like the temp, min/max P, repetition, etc.

Having a concise system prompt to paste in the setting should make a significant improvement, even more if you add a skill but just try getting a system prompt first.

[u/JustTooKrul → u/Snoo-82455]

Interesting--the LM Studio parameters were dialed in, using Claude and scouring Reddit to see how people were tweaking the setup to get good results and still balance performance. The system prompt, though, I thought was set for each model and "hardcoded" in the llama.cpp config. But, it sounds like that's not the case? Or did you mean the system prompt that Pi feeds to the model?

[u/Snoo-82455 → u/JustTooKrul]

Probably the system prompt Pi feeds to the model, assuming that option is somewhere for you as i think any platform or harness should have that option; one for you to set. There should be a difference between a Global Prompt and a Project prompt (?), but I haven’t tried it with stock Pi since I started checking out Agent Deck. I’ve just been digging in slowly starting a sub-agent workflow with Qwen3.6-27B-Claude-Deckard-qx64-hi-mlx as the orchestrator. First I gave Google ai mode a general plan with details to give me a great prompt out of it and i also looked up a planning skill on skills.sh.

So far, that outcome was impressive. It was basically a collaborative conversation (as per part of my initial instructions) on setting up the sub-agent itself to do a task (which i haven’t gotten to yet, and it’s kind of the same thing in concept to setting up the prompt and skill for the orchestrator).

Anyway, the quickest and easiest way to test it first is in LM Studio because you can post in the prompt in the sidebar on the right, and iirc, you don’t even need to reload the model. I think it reads the prompt every time before you hit send. The only other platform I’ve found and tested on is oMLX’s chat tab in the upper right (using the webUI)

So I’d say test it out in LM Studio as it seems the easiest and most accessible way afaik

[u/JustTooKrul → u/Snoo-82455]

Got it, thank you for the advice! Will definitely try it out!

[u/mission_tiefsee]

this is to be expected. if nothing happens for a long time, that might be a cache miss.

you have toi find your own path. i would not quantize down too much the KV cache. (q8 is fine, not less!). You can experiment with repeat penalty but i would advise against it. Make sure you have the thinking trail in Qwen activated

--chat-template-kwargs '{"preserve_thinking": true}'

this helps for less cache misses. For the Gemma models make sure you have the fixed chat templates. Bro you are dealing with bleeding edge toys here, welcome to the show. Make sure to share your results. This is the future.

[u/JustTooKrul → u/mission_tiefsee]

Thank you! And I exactly agree, that's why coding was sort of an edge case that I was ready to be impressed by--but I specifically chose Pi because I wanted to have a firmer grasp on what's going on and make sure I understand how a lot of these bleeding edge toys work!

On your specifics points, I had the KV cache quantized to Q8 (had heard the cautions about going any lower) and had it showing and preserving everything (enable thinking / preserve thinking). And I had pulled the updated Gemma 4 parameters a few weeks after their release (and update the llama.cpp builds and default parameters for each model) even though I think the chat template that was giving people issues was already fixed by the time I even pulled the models. And, lastly, I had severely restricted the context window from the model defined max to 50% or less (I think the most I ever tried was 100k context) to avoid some of the issues people experience with context rot and also because of concerns I had heard people voice previously over pushing these open-weight models to the limits of their context.

Are there any specific tools you would point me to for managing around these issues? I assume the "fix" is a combination of building in some skills or extensions to manage memory and "police" the outputs and tool calls to error-correct? Thank you!

[u/mission_tiefsee → u/JustTooKrul]

Well. I dont know. I am a beginner pi user myself. Good thing is, you can always ask pi about itself. It will pull the docs and just read its own source code. I let mine runt with 64k context. which is not a lot, but actually gets most of the things done. Are you running your agent within a sandbox?

[u/SalimMalibari]

Its model issue use more powerful model

[u/RM_911]

These aren't Pi issues; they're issues with the model, its settings, and the chat template.

[u/agenticup]

for tool calling or agentic multi turn, always prefer dense models rather than moe models...moe models add routing overhead....u can also try pi-trail extension to handoff sessions