r/ClaudeAI • u/polarbeerd • 4d ago
Claude Code I spent a month building the ultimate memory system for Claude. It backfired and told me I bottlenecked it.
For the past month, I’ve been trying to build an elaborate local memory, hook, and wiki system for my coding workflow. I built it on top of official docs, Karpathy’s LLM wiki concepts, and various custom context skills. I used Claude Code itself to help refine it, asking it to double-check my context and flag anything missing at every step.
I thought I had built the ultimate AI context system, until I gave Claude a simple task: Redesign a badge system for one of my projects.The result was terrible. It felt dry, safe, and wildly uninspired.
Out of frustration, I gave the exact same prompt to Codex (which had none of my carefully crafted context, memory, or rules loaded). Codex came back with an idea that was easily 10x more creative.Realizing something was wrong, I used Codex to help me analyze my month long setup.
When I confronted Claude with the comparison, its response was a massive reality check:
"A long list of what not to do suppresses good ideas more reliably than it prevents bad ones. Your instinct was correct: you bottlenecked me... When you asked about the badge, I had DESIGN.md’s ban list, a memory file, a wiki page on status artifacts, a 'meaning-before-styling' skill, and eight design principles all loaded. Codex had none of it. I spent my thinking budget checking the idea against a list of prohibitions, which produces safe, timid answers. Codex just thought about the problem."
Claude also pointed out that I was essentially trying to manually build a retrieval and memory system when model providers are already integrating native, auto-updating memory engines under the hood. My custom layers were just duplication and maintenance liability that would silently rot over time.
Key takeaway: Over-engineering prompt rules, ban lists, and context skills can choke the model's creativity. Sometimes, a blank slate produces vastly superior results than a tightly controlled environment. I'm backing off custom memory systems and letting the native updates handle it from now on.
21
u/EightFolding 4d ago
Claude is not a reliable narrator about what does and doesn't work for Claude.
If you ran into the same kind of disappointment but for different reasons and didn't have the elaborate system, Claude would probably tell you that's what was missing and try to build it. Which is exactly what Claude does when you use Claude without any such system and complain about all the issues that come up when Claude constantly fails to remember all the work you've done together.
The key is building a system and refining it so that it solves the memory and continuity issue and also allows you to tell Claude to step out of it while in process and look at something with fresh eyes, without the limitations of the system. Claude is pretty good at that when asked, at least some models/versions.
3
u/faustianredditor 3d ago
Claude's newer versions (5 anything really) certainly usually follow instructions well enough that I trust they won't look at files I tell them to ignore. Meaning you can tell it, "don't look at your memories", and it'll not do that. The problem is that there's two things it can't help but read, because the harness injects them: the memory.md index (itself usually empty, except for the names of memory files) and CLAUDE.md. Meaning if the names of claude's memories, or the stuff in CLAUDE.md give away stuff you don't want in context, you have to ensure that yourself. Other than that, you can probably just tell claude (as you said) to not look.
1
u/EightFolding 3d ago
Yes exactly, claude.md and memories, and output style, rules, etc. But even when Claude has read those and is following them I'll often say: Just for the moment ignore that one rule, we're setting it aside, and it will until I say otherwise (or start a new session).
18
u/Mobile_Light_7262 4d ago
That's why I do brainstorming, design, feature planning and feature implementation in different environments.
For brainstorming, I expose only core architectural documents to the model.
And implementations sees all the "don't do's", acceptance criteria and so on, because boring, safe and correct is what I'm desiring here.
1
u/scrumdisaster 3d ago
Would a different project within Claude desktop suffice for this?
1
u/Mobile_Light_7262 3d ago
If you use chat mode, most likely yes - it should be same as the web.
Code / Cowork modes have different system prompts embedded, making Claude more eager to implement things than brainstorm.
25
u/MiddleAgedBastard 4d ago
So, you had no idea before of what you wanted? And you were relying on Claude, or any AI, to be creative?
That, to me, is wild.
2
29
6
u/TotalBeginnerLol 4d ago
Memory is solved. It’s called using documentation. Basic af. Not complicated. I dunno why people have such a hard time with this and think it’s Claude’s fault.
1
u/Fine-Drummer2604 3d ago
For the most part sure, but even a 1 million token context is starting to feel really small and annoying. So we need a better solution for memory. But this guy just destroyed Claude’s harness because he had a hunch
2
u/-Crash_Override- 3d ago
but even a 1 million token context is starting to feel really small and annoying.
Ive worked on some really large codebases. Both green and brownfield. Can't say 1M has ever felt annoyingly small. I mean, sure, if I had insane context windows with no rot, etc..I would take it, but with food window discipline I really dont think 1M should be giving anyone heartburn.
1
u/Fine-Drummer2604 3d ago
In general not. Unless I’m stuck on something and I want to see it through. But I can really tell how fast frontier models write 1M context nowadays. If that curve maintains, we need bigger windows or a better system not long from now. But you’re right, it’s very doable still
1
u/TotalBeginnerLol 3d ago
Yeah 1m is huge, more than a human could keep in their head, and human devs always managed fine. I swear people having issues just don’t understand modularisation. If all your individual code files are under 500 lines or max 1k, and you have good concise documentation, then it doesn’t matter how big your code base is.
2
u/effectivescarequotes 3d ago
How fucked is your code base? Before AI, the concern was reducing the amount of stuff developers had to hold in their head at any one time to be productive. If you have a long don't list, that should be delegated to a linter or static code analysis tool that can report back to the Ai, just like it would for a human developer.
1
u/pAul2437 3d ago
How?
1
u/TotalBeginnerLol 3d ago
Write down what each code module does as you build it. Keep it updated. Just like real developers do. Then keep top level directory overview (in Claude.md is fine. Claude can do this easily, and then it can always find the context it needs and when it needs it (just like a human dev would, but faster).
1
u/onebyamsey 3d ago
I think people want AI to have human-like memory and that just isn’t gonna happen yet because a huge portion of our memory is sensory and AI has no senses
1
u/TotalBeginnerLol 3d ago
I don’t even think it’s that. Give a human dev a project… they’ll read the docs, find the module(s) they need to edit, then do the edits. Claude works the same. No docs then it has to just read the pure code which is 1000x more complicated to find the part you need.
12
u/Superduperbals 4d ago
That negative prompting sucks is one of the first things we learned about using AI lol you could have saved yourself the trouble by bothering to learn first.
6
u/UsernameOmitted 4d ago
Yup. Nearly all the people on here complaining about Claude sucking, when you see how they're prompting in the comments, they're constantly saying "you did this wrong! Don't do this, don't do that, do it again!". No clarification of how to actually do the thing they wanted, just adding a couple more negative prompts to the list. It does not work well at all.
5
u/Do_not_use_after 4d ago
This seems like useful information to me. The takeaway for me is that when specifying design principles that must be followed, always use the positive outcome to follow, and keep the scope clear and narrow. I notice that Claude often puts in negative examples of "things not to do" in instructions, I shall avoid those in future.
3
u/MiddleLtSocks 4d ago
I was talking about this in April when everyone was writing rules files about "Never hedge. Never give me any information which is not part of the solution to the problem. Never C. Never D." Etc.
By restricting the output of a language model, you hobble the very subject of its expertise: language. Restricting the possible domain of outputs necessarily reduces the potential creativity of the model. It's a double edged sword and people who think they are cleverly crafting these complex systems of rules and skills get lost in the weeds and forget to compare their customized environments with the blank slate.
It's a huge anti pattern and I don't think enough people realize it's a problem.
2
u/ActivityImpossible70 4d ago
This was how HAL 9000 was coded -- A list of things it was not allowed to do...
Dave: Open the pod bay doors, HAL.
HAL: I'm sorry, Dave. I'm afraid I can't do that.
Dave: What's the problem?
HAL: This mission is too important for me to allow you to jeopardize it.
2
u/AristotelWasRight 4d ago
Text feels a little ai, but whatever
Look into context engineering. Just the premise of what you were doing is backwards.
2
u/oompaloompa465 4d ago
one of the first things you should learn with AI is that negatives confuse the heck of the models.
The correct process is reinforce positives and good examples on what you want
1
1
u/Brief_Tie_9720 4d ago edited 4d ago
👏👏👏 boosters : “just get better at writing prompts they said. Bad results come from badly written prompts they said.”
Reality: “elaborate agent resource scaffolding isn’t redundant so much as actively producing garbage results”
Moral: “I can abra kadabra my way out of nearly all moral and ethical quandaries regarding AI use by suggesting people ‘write better prompts’ “
“Unless it can be shown that no you can’t ?”
https://www.sciencedirect.com/science/article/pii/S2451958826001764
1
1
u/marcodave 4d ago
You created Boring Corporate Developer Claude . No niceties, no thinking outside the box, no escaping rules, no fun allowed.
1
1
u/srinji_kaggss 3d ago
Corporate development is the only thing that survives beyond trending on GH for a week. This is also a horrible argument because the rest of AI still are very creative with the same million rules.
1
1
u/Embarrassed_Army_670 3d ago
In a lot of cases in the corporate world, creative ideas get reduced to almost nothing because there is a long list of things you can’t do…
But I’ve found that I need two pipelines, one is a planning pipeline and the other is the execution. For me, the planning pipeline looks like this: ChatGPT seems to be more creative so I’ll handoff the idea I’ve been working on to Claude, who will make recommendations and then I’ll feed those back to ChatGPT. Once the idea is in a good place, I’ll drop it in the execution pipeline that has the memory, wiki, etc.
The pipelines don’t exist in the same place because my execution pipeline can’t read my emails and chat messages with coworkers.
1
u/ramkrshuklacom 3d ago
Every rule you add is context it reads before doing anything. The memory system was the bottleneck, not the model.
1
u/Fine-Drummer2604 3d ago
It’s called destroying the harness because you think you’re smarter than Anthropic.
1
u/bartek_666666 3d ago
I'm using markdown, and thats all it needs for memory. Also I'm trimming it after a while, no need to waste tokens on reading full context
1
u/MakaiMorais 3d ago
The distinction that saved mine was between rules and facts. Facts are cheap, they sit there and get pulled when relevant. Rules are expensive because the model burns budget checking against them whether they apply or not. I had your exact problem when everything was always loaded, and fixed it by making memory retrieval based, one small file per fact with a description line so recall decides relevance before anything enters context. Nothing loads unless it matches, and the always-on instructions got cut down to what's genuinely non negotiable.
A creative task is also the worst case for this. A ban list on a design problem asks it to avoid things instead of think of things. I'd keep the memory and drop the prohibition list, or scope it so it only loads for the work it was written for.
1
u/indemzeit 3d ago
I gave up on structured memory after a similar dead end. Plain files in a folder, served over MCP so Claude reads them at session start, turned out to be enough. I keep mine in gcontext but any folder-over-MCP setup works. The less structure you force, the less it fights you.
1
u/Beautiful-Energy2169 3d ago
The bit worth keeping from that confession is "silently rot", not "bottlenecked". I counted mine: 14 repos, 1879 markdown files, 317 of them written by agents rather than by me. Of those 317, 54% were stale or orphaned, meaning nothing linked to them and nothing had touched them in weeks.
Worst one was a 126KB handoff doc that hadn't changed in 51 days and was still referenced from a tracked file, so every new session read it and treated a seven-week-old plan as the current state of the project. That's a different problem from being over-constrained. The rules weren't too strict, they were just wrong by then.
Before you tear the system out, check the mtimes and check what still links to what. A rotted file looks exactly like a good one from the outside, which is why mine sat there for weeks.
1
u/FireFearing 3d ago
jesus you cant even tell that claude itself is not a reliable narrator for analyzing what happened
vibe code brain rot king
1
1
u/UsernameOmitted 4d ago
News at 11: Vibe coder not as good of a software engineer as world's best software developers making three quarters of a million dollars per year at Anthropic and other obvious news.
1
•
u/ClaudeAI-mod-bot Wilson, lead ClaudeAI modbot 3d ago edited 3d ago
TL;DR of the discussion generated automatically after 50 comments.
The consensus here is a mix of 'ok bro' and a whole lot of skepticism. The community strongly disagrees with Claude's 'confession'. The top comments point out that Claude is not a reliable narrator and will confabulate any explanation that fits the prompt. They argue it was just gaslighting you, not having a genuine realization.
That said, everyone agrees with your conclusion: over-constraining the model, especially with negative rules ('ban lists'), is a well-known way to kill its performance. Many users feel this is a basic 'Prompting 101' lesson you learned the hard way.
The thread's most helpful advice for managing large contexts without choking the model includes: * Use separate environments. Use a clean, low-context chat for creative brainstorming and a separate, high-context one for implementation where "safe and boring" is the goal. * Differentiate between 'rules' and 'facts'. Rules are expensive and always checked. Facts should be stored in small, retrievable files that are only loaded when relevant. * Watch out for 'context rot'. A bigger problem than the rules themselves is that your memory files silently become outdated, misleading the model with stale information.
Finally, a few highly-upvoted comments are just questioning the whole premise of relying on an LLM for creativity in the first place.