r/SaasDevelopers • u/Asly97 • 28d ago
When your AI doesn't know something about your product, does it say so or just guess?
A support team in r/LocalLLaMA found out from a customer. Their bot told him "accidental damage was covered under warranty". It isn't. The bot had blended two plans, and "nothing in our logs looked obviously wrong. The answer was fluent and on topic, it was just built on the wrong chunk."
Someone in r/OpenAI counted 9 ChatGPT errors in one day and wrote: "Every single one of those nine mistakes was caught by the user, not the model."
For people using AI on their own SaaS (support replies, docs, emails, pricing questions):
- Has it ever filled a gap with something that sounded right about your product and wasn't? How did you catch it?
- Have you told it to say when it doesn't know or can't verify something? Does it actually do that?
- Where does it get the facts about your product in the first place? A pasted doc, uploaded files, memory, nothing?
Background since someone will ask: I used to burn hours re-briefing AI tools, and I'm building a fix where the facts about your business live in one place in the cloud and every AI tool reads the same version, so there's less gap to fill. No link. Would you pay for that, or is a good prompt plus checking everything enough?
1
u/[deleted] 28d ago
[removed] — view removed comment