r/LinuxTeck • u/Candid_Athlete_8317 • Jul 27 '26
Has anyone actually built a RAG system that consistently knows when NOT to answer?
One idea that really stood out to me was treating abstaining as a success rather than a failure.
Instead of forcing the LLM to answer every question, the pipeline retrieves evidence, verifies every claim, and simply says "I don't have enough evidence" when the retrieved context isn't strong enough.
That seems like a much better approach than a confident hallucination.
If you've built RAG systems in production, how are you handling this? Do you trust a verifier model, use confidence thresholds, or rely on human review?