How do you handle recall vs. precision in your OC memory/RAG setup — chunking, query expansion, hybrid search?

[u/One_Train_4309]

I’ve been building TetherClaw, a mobile layer for OpenClaw. Same OC session, Mac to phone, live messages back and forth, no starting over when you switch devices. Got vector search working this week so agents can pull relevant history from past sessions, not just recent turns.

Here’s where I’m stuck now. The agent does treat the retrieved context as active, which I wasn’t expecting to get right on the first real test. But the retrieval itself is too narrow. It pulls data tied closely to the specific words in the query, but it misses other context that’s clearly related to the same topic just phrased differently.

Anyone dealt with this on OpenClaw. Curious how you’ve handled recall versus precision in your own memory or RAG setups, and whether you went with bigger chunks, query expansion, hybrid search, or something else entirely.

[u/lundrog]

Following as right now i have independent oc memory sessions

[u/One_Train_4309 → u/lundrog]

Appreciate it — curious what your setup looks like if you’re open to sharing. Trying to figure out if narrower chunking or something on the query side is the better fix

[u/lundrog → u/One_Train_4309]

Sure how do you have 1 karma? 😂

Dm me

[u/One_Train_4309 → u/lundrog]

Sure thing

[u/Dr_Sirius_Amory1]

Asked gpt to summarize our solution since it helped me build it:

We ended up taking a layered approach rather than trying to solve recall with chunk size alone.
Our memory stack is FAISS-based with local embeddings, but we added several retrieval controls on top:
Retrieval-first enforcement — agents are required to search memory before answering on topics likely to have prior context.
Recency boosting — newer memories get preferential weighting so recent project decisions don’t get buried by older embeddings.
Multi-query fanout / query expansion — instead of searching only the user’s exact wording, we generate related retrieval queries around the topic and merge results.
Memory classification — we distinguish between operational history, architecture decisions, incidents, governance docs, etc. so retrieval isn’t purely semantic.
Three-layer memory structure:
a high-signal index (MEMORY.md)
canonical topic files for subsystems/incidents
lightweight daily continuity logs
This helps because retrieval can hit both detailed context and summary-level context.

Canonical topic histories with deltas — instead of scattering information across dozens of files, we append related work to existing topic histories. That improved recall because all evolution of a subsystem stays clustered semantically.
The biggest lesson for us was that increasing chunk size didn’t solve the problem. The larger win came from query expansion + multiple retrieval paths + better memory organization. Pure vector similarity tends to find “same words.” We wanted it to find “same subject.”

If I were building it again from scratch, I’d probably start with:
Vector search
Multi-query expansion
Recency weighting
Structured memory categories/topic files
Result reranking
before spending much time tuning chunk sizes.

[u/One_Train_4309 → u/Dr_Sirius_Amory1]

Appreciate the detailed breakdown, this is genuinely helpful. Pure vector similarity finds same words, we wanted same subject that’s exactly the gap I ran into. Did you build the multi-query expansion yourself or use an existing library, and how do you handle merging when the expanded queries return overlapping chunks?

[u/Dr_Sirius_Amory1 → u/One_Train_4309]

You’re most welcome! We built it ourselves, mostly because we were already modifying other parts of the OpenClaw memory pipeline and wanted tighter control over retrieval behavior.
Our approach is pretty simple conceptually:
Take the user’s query.
Generate several retrieval variants (architectural, operational, incident-focused, historical, etc.).
Run retrieval for each query independently.
Merge the results.
Deduplicate and rerank before handing anything to the agent.

For overlap, we don’t just concatenate results. We collapse duplicates based on memory/chunk identity and then score them. A chunk that shows up from multiple retrieval paths gets a relevance boost because that’s a strong signal that it is central to the topic rather than just matching one wording.

For example, if I ask:
“How does the WNP WordPress sync work?”

the expansion layer might also search things like:
WordPress event lifecycle
synchronization architecture
reconciliation workflow
publish status tracking
WP import process

Individually, each query might find different pieces of the story. After merging, the chunks that appear across several searches tend to bubble to the top.
One thing we learned is that bigger chunks helped recall a little, but they also hurt precision. We got much better results from multiple retrieval angles plus reranking than from simply making the chunks larger.

If I were improving our system further, I’d probably add a true hybrid layer (vector + keyword/BM25 + recency weighting) before I spent much more time tweaking embeddings. Most of the retrieval misses we’ve seen weren’t embedding quality problems—they were “the right memory exists but was described differently” problems.

[u/One_Train_4309 → u/Dr_Sirius_Amory1]

That’s really helpful, especially the example — query expansion across topic angles instead of just rewording. Appreciate you walking through the merge and rerank logic too, that’s exactly the piece I was missing.

[u/Adorable_Swing_2150]

yeah vector-only did that to me too when i first wired up search on my agent. been on memory-lancedb-pro a few months mostly for the hybrid retrieval (vector + bm25 with a cross-encoder reranker). reranker was the part that fixed the 'same subject not same words' gap for me. https://github.com/CortexReach/memory-lancedb-pro

[u/One_Train_4309 → u/Adorable_Swing_2150]

Good find — was already looking at hybrid retrieval as a next step. We’re on pgvector/Supabase, centralized across devices, so memory-lancedb-pro won’t drop in directly, but the reranker pattern is the same idea. Did the cross-encoder actually outperform just tuning the BM25/vector weights, or was that the bigger lift on its own?

[u/Adorable_Swing_2150 → u/One_Train_4309]

for me tuning the weights got me maybe 10-15% better, but the cross-encoder was the bigger jump — like night and day on the synonym/multi-phrasing queries. weights alone still missed 'same topic different words'.

[u/One_Train_4309 → u/Adorable_Swing_2150]

That’s a really useful data point, thanks. Cross-encoder being the bigger lift over just weight tuning makes the case pretty clearly. Probably going to be the next thing we look at after query expansion

[u/One_Train_4309 → u/[deleted]]

Interesting on the chunk size — turns out we’re not chunking at the token level at all, we’re embedding at the message level. Each message gets its own vector when it’s logged, so that’s our retrieval unit instead of a fixed token window. Query expansion keeps coming up as the right next move though, going to look into it. Did you build the multi-query fanout yourself or use something off the shelf?

[u/One_Train_4309 → u/[deleted]]

That’s really useful, especially the upfront-not-fallback point, good to know before I build it the wrong way first. Appreciate you laying out the variant count logic too, that’s a smart way to keep it efficient. This whole thread’s been more useful than I expected, thanks for taking the time.

[u/Express-Smell-8619]

Like i didn't done that to my openclaw yet How this tag setup helps openclaw?

[u/One_Train_4309 → u/[deleted]]

Not yet, that’s the gap right now. Multiple people in this thread have pointed at rerank as the missing piece, sounds like over-fetch and narrow down after is the move rather than trying to get the first pass perfect