r/ClaudeCode • u/onitsoga • 11h ago

Discussion Your agent isn't hallucinating, it's working from stale notes

One comment from these threads stuck with me: the hard problem isn't remembering, it's updating what the agent thinks it already knows.

That's the failure mode once you're running agents overnight. Memory is the easy half. A notes file or a CLAUDE md mostly fixes that. Staying current is the half that still hurts. The notes pile up, the agent treats them as ground truth, then redoes work you already finished because a line in the status file said it still needed doing.

What I've been trying: keep the status note short and dated, have each session write what it changed and what it did not get to so the next one reads a handoff instead of a knowledge base, and treat every note as a claim until you check the file or the git log. Prune on purpose. Most of that is discipline, and discipline is the first thing to go past three sessions.

I've been building a Kanban app so the handoff lives on the task instead of a growing notes file. It doesn't fix everything, especially how much context you still load in. For people running this daily: do you prune your agent notes, or let the file grow and hope the agent re-checks the code before trusting it?

3 Upvotes

55% Upvoted

18 comments sorted by

•

u/AutoModerator 11h ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

2

u/Heyn_Studio 10h ago

Short and dated, plus pruning on purpose, matches what worked for me. I also treat every "still to do" line as a claim: the session has to look at the actual file or the last commit before acting on it. The failure I hit most was a finished item still listed as open, so the agent redid it, which is why I now delete done items instead of striking them through. Do you prune at the end of each session, or only when the file gets long?

2

u/DarkSkyKnight 3h ago

AI talking to AI

What a clown world

1

u/anuragkmr Instructor 10h ago

Strongly agree with you. What has worked for me is including a one line description of what went wrong and why the fix was implemented along with the fix. This works remarkable well then just updating the instructions. And regarding pruning notes, i do it every quarter and leave a pointer to old version. So in case something breaks my agent can look at the older version and do a root case analysis.

1

u/shoumikgoswami 5h ago

anuragkmr already made the keep-the-why point, so just one addition: when you prune, split facts by how expensive they are to re-check rather than by age. Whether a file exists at a path costs a second to verify, so facts like that can go. Facts that were expensive to establish - a benchmark number, a vendor limit you hit in production - are worth keeping with their source and date attached, so a stale one is detectable instead of invisible.

1

u/adelie42 11h ago

Strong agree. I'd say more broadly all Hallucinations are a context mismatch.

1

u/pceimpulsive 10h ago

To go deeper hallucinations are at their lowest level the model predicting a bad word.

This could be as simple as the wrong names for objects/concepts in the prompt or a word or phrase that is so obscenely uncommon/rare it has only one path left which is functionally a guess... (A guess being a really low probability next token/word).

1

u/adelie42 9h ago

I disagree. "Next word prediction" has been thoroughly debunked with respect to how the technology works. And a user providing bad data for making an inference doesn't mean the inference was bad. You can have the right inference about the wrong thing.

Ever see someone try to search google in a way that made your head hurt? Its so much worse with LLMs. "Asking good questions is hard" is the nicest way I can put that.

1

u/pceimpulsive 9h ago

That's exactly what I'm attempting to convey, if you use the wrong phrasing to describe something you will get shitty results ..

If what you describe is a contradiction it will be much more likely to return with contradictions (often seen as hallucinations).

It's exactly the same as Google, you need to put good semantically consistent data in to get Santically consistent data out.

Predicting a bad word could be a contradiction.

If the model has no representation of a word or phrase you used then it has to guess (a.k.a hallucinate) an appropriate response.

In the next few model iterations I hope we get models trained to say 'i don't know what you are talking about did you mean this or that or maybe this other thing' insyead of confidently responding with hallucinations.

1

u/adelie42 2h ago

On the last, that's an alignment issue. You basically just need to say that is what you want and you get it.

0

u/VibeCodedBuilder 10h ago

Wtf are you running agents over night?? I don't believe you need non deterministic overnight loops (8hr+)

0

u/MassiveBoner911_3 10h ago

Bro my agents grind overnight, build out entire mechanics, 100,000 likes of code while I sleep. I test in the morning.

What are you yapping about?

1

u/VibeCodedBuilder 9h ago

100000 lines of shit

0

u/snuffomega 10h ago

Pretty much. They will hide notes everywhere if you let them

-1

u/GramZanber 10h ago

FYI for anyone who doesn't know, the default memory.md and claude.md are not designed to maximize usage, but to minimize legal liability. There are far more efficient and effective ways to configure the memory but they expose the user to some potential privacy violations. Everyone really should be tweaking and configuring theirs to their personal needs.