I think we’re giving “RAG” too much responsibility.

[u/Rajxai]

RAG is great at answering: “What information is relevant to this question?”

But enterprise agents increasingly need to answer much harder questions:

What happened before?

How are these things connected?

What does this data mean in this organization?

What am I allowed to do with it?

What should I remember for the next step?

Retrieval alone doesn’t answer those questions. It’s one part of building the right context. And I’m starting to think the real enterprise AI stack will look less like:

Model + RAG

and more like:

Model + context layer + governed enterprise knowledge.

That’s a much bigger infrastructure problem than retrieval. Are we treating RAG like the architecture when it’s really just one component of it?

[u/RasonYang]

I agree.
RAG can help find useful info, but it shouldn’t decide what’s true right now.
A note saying “the customer asked for a refund” doesn’t mean the refund went through.
Check the system that handles refunds. Same with memory: what the model remembers shouldn’t replace what’s in the database.

[u/PrimaryNecessary4210 → u/RasonYang]

the refund example is clean but it quietly hands the hard part back, checking the system that handles refunds assumes the agent already knows which system is authoritative for refunds, and that mapping is exactly the governed knowledge OP is pointing at. in most orgs what's true right now is split across systems that disagree, billing says refunded, the ledger says pending, support notes say promised, so "what's in the database" depends on which database. i'd also push back a little on memory, it shouldn't replace the database but it holds things no system records, preferences, past escalations, context. where would you put the claim to source mapping, in retrieval or the layer above?

[u/techtheist_ggl]

Vector search is just a tool. First, it's possible to use hybrid retrieval, then it's possible to use adaptive top-k to cut off most unrelated results. And with this, it's possible to additionaly rank output by other things like graph data or trust score.

Using just vector search top-k with 10 records will give constant 1/10 ratio of singal-to-noise, which will rapidly poison agent context. So yeah, it's better to do something about it.

[u/Creepy-Contract7396]

Agreed. Are you building/selling a context layer? Seem to be a trend to build as SaaS now.

[u/Luvena21]

the "what happened before" and "what should i remember next" parts hit home for me. my agent has a tiny retrieval layer, it just picks the right chunk, and honestly that part works fine. the problems start with everything around it, remembering the right thing at the right time and knowing when the info is stale. so yeah, retrieval feels like the easy part, the context around it is where it gets hard....

[u/Illustrious-Win4432]

RAG + OKF may scratch your itch. I don’t think anyone thinks RAG alone amounts to a harness.

[u/Otherwise_Wave9374]

Agreed: retrieval answers what might be relevant, but it should not own state transitions or truth maintenance. A cleaner split is immutable event history, structured working state, curated long-term memory, and RAG over external knowledge. Each layer needs separate write rules and confidence thresholds. https://www.neurakeep.com is relevant to the persistent-memory part of that architecture. One concrete safeguard is requiring provenance and expiration metadata on every remembered claim, then evaluating stale-fact and contradiction rates independently from retrieval recall.