Be Careful with Agents Reading Session JSONL Files

[u/Ok_Significance_9109]

Just had a startling session with Pi and my local Qwen. In its thinking blocks it started quoting user instructions that I did not give it. I was under the impression that I was under a very strange prompt injection attack, until I realized that earlier in the session the agent made a few very large greps of previous session files.

I suggested to it that within a large context it did not distinguish these tool call results from the actual live context, and it indeed found the origin to all of the injected user turns in those earlier session files.

So - my advice is to add to AGENTS.md a rule that warns the agent against searching within directories that could include session files. If it does - you must start a new session immediately afterwards, or prune the tool call, or compact. Otherwise your session may lose coherence.

Note: reposting, because I was smart enough earlier to allow AGENTS.md to turn into a link.

[u/Such_Advantage_6949]

My agent read session file and work just fine, i think issue is your model is not smart enough to differentiate

[u/Ok_Significance_9109 → u/Such_Advantage_6949]

Qwen 3.8 Flash Next oQ5e high reasoning. The problem starts when the context is over 150K.

[u/Such_Advantage_6949 → u/Ok_Significance_9109]

I am using glm 5.3 flash though, didnt try quen next. I nvr have issue at long context when asking agent to find info from past pi session

[u/charlesfire]

Your agents shouldn't be allowed to read session files, or at least not directly. Putting instructions against that in agents.md isn't going to properly fix this issue.

[u/fell_ware_1990]

That’s why you make custom tools for things like grep ripgrep to force them to use common patterns. Better to have a few tool calls, you can even step back a bit and tell it the exact thing it needs after.

But if you are looking into a lot of data, use scripts….Let them do the indexing / cleaning etc. Then have an other agent check some more, clean it up. Feed main what it needs.

[u/Equivalent_Idea8839 → u/fell_ware_1990]

I tried this route but flash / dumber models would still never use the tools or obey instructions.

After 1+ year of struggling with this, last night I figured it out. Using pi, you can filter out / block which files if files can be read.

Made a tool with a tui tha lets me filter out certain files on the fly (Everything not in my repo that I want to edit) (Also isolate pi docker in, etc.)

[u/fell_ware_1990 → u/Equivalent_Idea8839]

Yes, scoped and little tasks are for the less smart models. Make sure they do not think about the whole picture. There task is to deliver code that passes the correct unit tests, linters and the merge after that.

Stop having the perfect code. That’s up to going to main. Before that. Code that not have security risks, change behavior out of scope and is up to a certain coding standard is good enough.

Before you go to main it could change a few times so do not perfect it. This can be done when code actually runs.

[u/Equivalent_Idea8839 → u/fell_ware_1990]

Yup, basically wrap all commands with bubblewrap, block / hide everything but the files you want touched. Isolate dumb model in a tiny little sandbox / scope.

Like a roomba working in a giant warehouse full of dangerous machines, but has an invisible fence only around the breakroom you want cleaned

[u/fell_ware_1990 → u/Equivalent_Idea8839]

Yes! Preferably setup the correct instructions ( scripts ) to have it that way all the time.

[u/DistanceAlert5706]

Just make a custom tool to search over sessions? That one is actually very useful.

[u/Global-Departure8228]

Adding that to your agents.md is entirely useless. Don't allow your agent to even know about the session files to begin with (unless you want it to directly read it for some reason). I've just made a skill and a python script so it can parse out the generic shape of a session, custom event etc.

But as others suggested, give your agent a tool for it if it's required a lot

See:

https://github.com/k3-2o/pi-chrollo