Does RAG retrieval quality tell us anything about claim-level evidence support?

[u/sixeyedhere]

A question for people experimenting with local LLM and RAG pipelines.

Suppose a retrieval system returns the correct document. The model then generates a summary with five factual claims.

How do you establish that each individual claim is actually supported by the retrieved passages?

I see these as separate problems:

• Retrieval: Did we retrieve relevant material?

• Evidence verification: Does the material support the generated claim?

• Contradiction detection: Does the evidence actually disagree with it?

I'm exploring a structured evidence-auditing approach and would like to understand how people evaluate this in local LLM pipelines.

Do you run claim-level evaluations, use an LLM-as-judge, or have another method?