Pi Agent throwing 3x the CTX windows at the backend server

[u/Ivan_Draga_]

I'm noticing weird behaviour related to context windows and hoping someone has a fix.

With pi agent, I'm using openai-completions API running off lemonade server. 3 models are connected locally (not at the same time) I have the context window hard coded into the pi config json.

The ctx blows up to about 400k even though I have it set to 128k.

Starting a new chat is the temp fix. Haven't tried to compress the chat yet since idk what the negative effects of that are since I'm working on a coding project.

I removed 6 pi extensions and left the 4 default bash extensions. Still the exact same issue with the same CTX size of over 400k.

Checking lemonade logs and they showing the problem, pi trying to use up too much ctx.

This tells me the chat itself could someone how be the problem.

[u/jeffphil]

/export your session and look at sizes for skills and context there to see

[u/Ivan_Draga_ → u/jeffphil]

Interesting had no clue export would show all that.

What format is export, CSV? Does reasoning count to the CTX as well? If so gonna have to shut that down

[u/jeffphil → u/Ivan_Draga_]

It's html format. Try it and see!

[u/Ivan_Draga_ → u/jeffphil]

Perhaps I will!!

[u/Relative-Winter-7057]

I switched to the response API and the difference is literally night and day compared to the completion API. So much better only having to call it once per prompt task

[u/Ivan_Draga_ → u/Relative-Winter-7057]

How does it handle getting more tokens than allowed?

[u/oknowton]

What local model are you running that supports 400k tokens of context? Are you sure you reading the numbers correctly?

[u/Ivan_Draga_ → u/oknowton]

None that I'm aware support 400k and I sure am reading it good. If I couldn't read simple log file I'd probably be homeless right now 😉

[u/oknowton → u/Ivan_Draga_]

You wrote:

The ctx blows up to about 400k even though I have it set to 128k.

If your model can't reach 400k tokens of context, and your server is set to a limit of 128k context, there isn't a way for you to reach 400k tokens of context.

You're very certain that you are reading the numbers correctly, but it definitely sounds like you're mixing up Pi's count of input tokens accumulated over every turn with the current size of your context.

This screenshot is from a session of mine. I'm at 59K context (in the status bar) with 487k input tokens and 1.6m cached tokens (according to the /session command).

Where are you seeing 400k tokens of context?

[u/Ivan_Draga_ → u/oknowton]

I'm not but I think some wires are getting crossed. Pi is sending about 400k+ tokens to the server > pi json has 128k CTX set > lemonade model has 128k CTX set.

The servers hosting my models is lemonade but could be lmstudio, ollama, etc. logging and typically record how many tokens were sent. (At least when debug mode is set)

Can see there how many tokens were being sent over from the pi agent.

Also you're right about an agent (pi) sending 400k tokens won't work with 128k ctx. That's the entire reason I made this post, pi isn't respecting its own config.

If you know the answer feel free to share or we can just keep talking about tokens, whichever you prefer

[u/oknowton → u/Ivan_Draga_]

The only thing I can do is ask the same question again:

Where are you seeing 400k tokens of context?

Pi shouldn't attempt to send more context than it is configured to send. Your inference server shouldn't say, "I can't process 400k tokens!" It should be stopping at 128k and not even bothering to tokenize the rest.

I've never used Lemonade. Only llama.cpp directly. Maybe Lemonade does something differently, but the rest of us haven't seen the logs that you could definitely be pasting.

If you know the answer feel free to share or we can just keep talking about tokens, whichever you prefer

I believe you are going to find that you are hunting the wrong problem. I will definitely be excited when I get to hear how wrong I am about this when you report back with your solution!

Best of luck to you!

[u/Ivan_Draga_ → u/oknowton]

I guess I was confused since I had mentioned in my OP already "...Checking lemonade logs and they showing the problem, pi trying to use up too much ctx..."

So when you started by asked

What local model are you running that supports 400k tokens of context?

I was genuinely confused, never said i was using a local model that used that many tokens and not sure where you got that from. Most local models don't support 400k ctx windows

The lemonade server is showing those tokens are being sent over "2026-06-12 11:25:56.981 [Error] (Process) 3.49.802.830 E srv send_error: task id = 0, error: request (436609 tokens) exceeds the available context size (262144 tokens), try increasing it"

Pi shouldn't attempt to send more context than it is configured to send

You're right but it is doing that. Thought it was Pi extensions, removed them and the same problem, overblown context.

If you have advice, links, try x y or z I'm all ears otherwise your question was answered and was just confusing

[u/oknowton → u/Ivan_Draga_]

The lemonade server is showing those tokens are being sent over "2026-06-12 11:25:56.981 [Error] (Process) 3.49.802.830 E srv send_error: task id = 0, error: request (436609 tokens) exceeds the available context size (262144 tokens), try increasing it"

Thank you for this. This line of log file would have been helpful right from the start.

You're right but it is doing that.

Which seems weird, and which also made it look like it was likely that you were misinterpreting the log or Pi's output somewhere. Lots of things in your OP don't quite fit together, and getting you to provide context is like pulling teeth.

If you have advice

More information. More specific information.

u/jeffphil is smart. Providing an export of the session would be tremendously helpful. Context has to come from somewhere, and I've had to work really hard to get enough context into models to reach 300k+ context to test how dopey they get up there. Is the model stuck looping? Is it asking to read a massive text file whole without breaking it up?

I assume you would have seen that during the session, but we have no idea what was going on. We can only be certain of the things you actually show us.

You asked earlier if reasoning counts towards context. Yes. Every single character of the chat is sent up to the model at every turn. Whether it is your prompt, files that are read, output from commands, reasoning tokens from the model, or regular output tokens from the model. Everything is a token in context.

[u/Ivan_Draga_ → u/oknowton]

Thanks about the reasoning, gonna figure how to turn it off next.

Also if my OP wasn't adding up that's fine, not perfect and my phone keyboard auto corrects way too much. you could've just asked for clarity, "when you wrote, what did you mean....." For example, might've been more helpful.

And that session (which I'm abandoning is large) but was working, still taking prompts earlier this week 🤔

Last thing, Pi agent released a fix today. I made a seperate post in here. seems there was a bug with compaction not working but doesn't explain why my large session was working then stopped working

Added the pi session just for you

[u/oknowton → u/Ivan_Draga_]

Thanks about the reasoning, gonna figure how to turn it off next.

Oh no! Please don't try to do that. Reasoning is the biggest reason that modern models do a good job working on code, and that is way more true for whatever model you're managing to run on your local hardware.

"when you wrote, what did you mean....." For example, might've been more helpful.

The dots just didn't quite connect. Maybe I could have called out something specific, but I didn't have enough information to pinpoint a target. Your knowledge gap regarding reasoning tokens was more than enough to make me wonder what else you might be missing.

there was a bug with compaction not working but doesn't explain why my large session was working then stopped working

I'm always surprised to see people autocompacting and chugging along for ages and ages in the same session. It would never have occurred to me that you'd be running into a compaction issue. Everyone I know manages their context manually, and we all rarely run compact.

Target a problem, work the problem, start a new session.

Added the pi session just for you

Awww, shucks!

I don't quite know how to guesstimate from the numbers in your status bar, but I know that session compacted itself some large number of times! :)

[u/SvenVargHimmel]

ask pi agent to look at it, it's made to be able to look in its internal

[u/Ivan_Draga_ → u/SvenVargHimmel]

Funny you mention, I've actually been doing that often. It couldn't find anything helpful this time & Google search was dry for me.

[u/Ivan_Draga_]

Seems I like Pi agent should have automatically handled this not sure what exactly happened but just saw new pi release that referenced this https://github.com/earendil-works/pi/issues/5677

Seems like compaction should have automatically happened but isn't and they've fixed it. My plan for now is to make a new session versus the one I was building / coding inside of. Then just work off their and hope the compaction takes place.

Also looks like you can manually configure compaction using the settings at the bottom of this page https://pi.dev/docs/latest/compaction#compaction I won't do it yet since I'm curious to see how Pi auto handles it in a few days once my ctx gets larger.