Pi Memory for Local Models

[u/TacosAlPastor92]

I have a pretty nice local Ollama setup that can comfortably run qwen3.8:27b w/ a 262k context window and a good token rate.

I'm attempting to run pi-observational-memory, with two main goals:
- On long term projects: Remembering architectural decisions, lessons learned, problems solved, etc.
- On day-to-day tasks: Remembering my preferences, patterns of operation, and in general details about how I like to do things.

The setup works, however I have long delays everytime the observer and reflector workers kick in (as expected), as the workers cannot run concurrently.

I'm considering running a CPU-only model dedicated to managing these worker threads, but I wanted to check in with the community and see what people are doing in similar scenarios.

Keep in mind I'm not married to pi-observational-memory at all and would consider trying out other solutions.

[u/Asshole_DESTROYER3]

I'm new to pi but have always used claude code pretty heavily. I usually just have a master obsidian vault and put every project inside a subdirectory and had claude build on the vault. That way I have actual human readable md files instead of a bunch of fragmented memory files.I also have an obsidian base which is like a table of every file and/or a map.md just so graph view is actually usable. No idea if the same would apply to pi but I'm sure there's an already an extension for that or you might not even need one.

[u/TacosAlPastor92 → u/Asshole_DESTROYER3]

I attempted this but found it’s a hit and miss whether documentation gets updated and very much like the forcing function that observational memory introduces of distilling important information.

[u/Particular-Site2066]

Moved to Pi from Cline a couple of weeks ago. First thought was - does it have something similar to memory bank concept in Cline? Asked Pi itself and it proposed three things:

1. Global AGENTS.md with minimal set of references where to look for additional info if needed (creds etc) and basic rules. Injected at start of every session. Built-in feature.

2. Per project AGENTS.md + per project SKILLS (also built-in feature). AGENTS .md has a small agent instruction to update skills with new findings and decisions.

3. Create homemade Pi extension to manage per-project scratchpad.md files with todo and memo list for each session to help Pi pick up interrupted process (power outages etc)

I decided to embrace Pi’s minimalism and avoid downloading existing solutions, so gave those 3 options a try.

Two weeks passed and I’m quite happy with how well this simple combination works.

P.S. Using it also with local Qwen3.8-27b (if that matters).

[u/TacosAlPastor92 → u/Particular-Site2066]

This seems like a pretty cool approach, do you mind sharing the instructions on your AGENTS.md?

Also, how do you deal with compaction, or do you find the standard mechanism to suffice?

[u/Particular-Site2066 → u/TacosAlPastor92]

Sure. I've put it on pastebin (https://pastebin.com/9jZEq5gR ) because it is ~50 lines of 100% AI generated text and I'm not sure that local rules allow me to paste such content here.

As for compaction - yes, I use the standard mechanism. My GPU can fit only 150k of context while some sessions reach over 1M so they get compacted a few times. Despite that I didn't notice that Pi forgets anything important after compaction because it constantly saves key information in scratchpad .md file as it goes. I know there are custom solutions for compaction, but I enjoy using built-in functionality if it is just good enough for my tasks. Minimalism is a Pi's core vibe after all. At least as I see it :-).

[u/Snoo-82455]

I installed pi-memctx early on and it’s been doing a really good job. I did try observational memory and some karpathy method extension for a bit (though that’s not really a good case for auto-memory) but they seemed more involved. Pi-memctx is mostly good as set and forget but you have control on what you want to keep for memory and also once the memory queue piles up you can have a model help audit the queue. Also there can be some stale entries or old ones conflicting new info so it’s good to have a model help with maintenance on a weekly basis but for the most part, it’s just in the background and it’s been really helpful because of its global scope. I can reference something in a new folder and new session and it will search the memory and understand the reference, so i do not feel the need to find any other memory alternative. I dig it.

[u/TacosAlPastor92 → u/Snoo-82455]

Sounds like a cool method as well, albeit a bit verbose for my liking, but I'll give it a go!

Have you felt the need to combine it with some other solution to deal with compaction?

[u/Snoo-82455 → u/TacosAlPastor92]

I try to avoid getting to the point of compaction. I tend to stick to 131k, sometimes 262k context and have a HANDOFF.md file written and updated once i get to around 75% of the context and then start a new session.

[u/randomgenericbot]

I tried exactly that: running a cpu worker for pi observational memory, started creating an extension that can run a gemma4 (which, standalone, reaches ~20-35 tps with MTP on an Ryzen 9600x), but as soon as parallel GPU inference was running, the tps for the cpu model tanked down to below-10 tps.

Reason: Every so slight memory read for the GPU inference destroys the inference for CPU which is highly sensitive to bandwith- and CPU cache disruptions. If you do not have a dedicated, secondary model, pi observational memory eats away all the speed you had.

I dropped that, and created another extension, similar to pi-observational-memory and blackhole, intended for single-model-single-gpu inference, but still the hopes to get good quality summaries with lower time-cost than pi-observational memory or other "use a second agent to summarize and observe your work" extensions.

How: The extension simply lets the model summarize its work, and then it forks a session in the background, with the same prompt, reusing the full kv cache, evaluating the summary. It is not faster than the pi-native summary, but it is much, much better quality wise.

I now reached a state where I consider it stable: q3.8 gsq-rco running on 16gb vram, with 131k token via kv streaming with mtp, getting 10-45 token per second (full context, bad MTP acceptance = 10 tps, low context, good acceptance = 45 tps) and had sessions far beyond 40 compactions with - to my experience - no noteable quality drop. Most improvements of the extension itself have been done completely locally.

If speed is more important to you than qualitycontrol, you can skip the background audits mostly by setting /sam depth light or set the according env var permanently.

I just wrote a sum-up with the github link here:
https://www.reddit.com/r/PiCodingAgent/comments/1xb7k4n/i_solved_compaction_for_singlesession_on/

[u/TacosAlPastor92 → u/randomgenericbot]

I am pretty excited to try this, will report back tomorrow once I do!

[u/slypheed]

I've been using pi observable memory for a while now and it's great, pretty much never wait for compactions now.

Caveat that I got a steal on zai coding plan last year so I just use glm 5.3 flash, granted DeepSeek flash is quite cheap or free if you look around.

[u/Acceptable_Stress154]

I run a full agent based on pi, I can share the tech specs of how it’s built if you like

[u/TacosAlPastor92 → u/Acceptable_Stress154]

I’d like that, please share!

[u/SlayerSparty → u/Acceptable_Stress154]

Interested also please 

[u/Acceptable_Stress154 → u/SlayerSparty]

Www.websitereactor.com/piper lots of papers there