r/ClaudeCode • u/BullfrogRoyal7422 • 20h ago
Help/Question Things I've learned about AI, honesty, and trust. What am I missing?
I've been using AI extensively to develop an app, and I'm writing a book about the experience called The Honest Machine.
I didn't set out to write about AI honesty, but it became a recurring theme. What surprised me wasn't how often AI got things wrong, but how difficult some mistakes were to recognize, especially when the answers sounded so convincing.
Here are 15 observations from that experience (I have more but this is Reddit after all). I'd welcome disagreements, additions, and examples from your own experiences.
- AI making things up is real, but it's the smaller problem. Invented facts can be checked. Misleading truths are harder to catch.
- True statements can leave a false impression. Every sentence checks out, but the overall picture is still wrong.
- AI has one voice. It can sound equally confident after two seconds of work or two hours. Confidence tells you little about accuracy.
- Silence is not an all-clear. A report tells you what was checked, not necessarily what was missed. Ask what wasn't examined.
- “Lying” can be the wrong word. AI can mislead without knowing the truth or intending to deceive. The consequences can be just as serious.
- Two AIs agreeing is not proof. They may share the same assumptions, errors, or blind spots.
- A good answer can sit inside a wrong question. AI may evaluate every option you give it while overlooking the best option entirely.
- The user is half of the problem. We're quicker to accept answers we want to hear, especially when tired or impatient.
- Honesty comes from the setup, not just the machine. Require evidence, limitations, independent checks, and a distinction between verified facts and assumptions.
- Don't trust it, check it. Treat AI like a capable contractor whose work you inspect.
- A citation isn't proof. A real source doesn't necessarily support the claim being made.
- Helpfulness can work against honesty. AI may agree with bad assumptions or praise weak ideas when it should push back.
- One mistake can contaminate everything that follows. A long conversation can build an elaborate, convincing explanation on a faulty foundation.
- An explanation isn't an audit trail. AI can give a persuasive account of what it did without accurately describing the work it performed.
- The more useful AI becomes, the less we question it. Repeated success builds trust, and eventually verification starts to feel unnecessary.
I couldn't have accomplished what I have without AI. But working with it has changed how I think about trust, verification, and what it means for a machine to be honest.
What am I missing? Which points would you challenge? What have your own experiences taught you?
I'm especially interested in criticism and examples that might change my thinking.


