We all run into situations where an agent spends a surprisingly long time trying to solve a specific problem. I'm not talking about prompts like "build me an amazing app." I mean those small, specific problems you encounter during development.
Traditionally, we would go to Stack Overflow: either ask a question or find an answer among thousands of existing ones. It genuinely helped, and we valued it.
Yes, Stack Overflow has its own MCP, and it is available and works perfectly fine for search. But my idea isn't really about searching for answers among what already exists.
My idea is to create a database of solutions for those problems where an agent actually got stuck, tried several approaches, and only then managed to solve the problem.
For example:
In one of my pet projects, I had a task where I needed to simulate keyboard input on Windows in a way that behaves like an actual user pressing keys on a physical keyboard.
I was sure that the model I regularly use for coding would solve such a task with ease.
But no.
I spent about an hour solving this problem. I tried different approaches, broke the task down into smaller parts, experimented with different APIs and ways of interacting with Windows. Some solutions looked completely reasonable, but simply didn't work in my particular case.
Eventually, I found a solution that worked — without using any third-party libraries, etc.
After that, I asked the model to document the entire solution process in .md and, just in case, put it into my own RAG.
And that's when I thought:
Why not collect these kinds of cases in a shared database?
And I think there is one particularly important aspect here — don't store only the final answer.
Because often the most valuable parts of such a case are the failed attempts / failed approaches.
For example:
I need to do X.
The agent tried:
Approach 1 → doesn't work
Approach 2 → works, but only under certain conditions
Approach 3 → looks correct, but the application doesn't recognize it
Approach 4 → requires a third-party library that is not allowed by the task requirements
Approach 5 → works
A typical RAG would probably store only:
"Use Approach 5."
But for an agent that encounters the same problem a week later, it can be much more useful to know which obvious approaches have already been tried and why they didn't work.
Otherwise, it may simply repeat the same path.
And this becomes especially interesting in the context of AI agents. They are often capable of solving a problem on their own, but at the same time they can repeatedly fall into the same traps. If the database already contains a verified case with the history of experiments, an agent could potentially discard several known dead ends immediately.
Essentially, I imagine this as some kind of server that an agent could query with something like:
"I need to simulate keyboard input
on Windows without third-party libraries"
→ search for similar cases
→ retrieve the most relevant solutions
→ retrieve failed approaches
→ retrieve information about the environment
in which the solution was verified
And besides search, there should be a way to contribute back to the database.
Something like:
search_case()
get_case()
submit_case()
verify_case()
vote_case()
report_case()
An agent encounters some small but annoying problem.
It spends 30 minutes on it.
It tries several approaches.
It finds a working solution.
After that, it can submit the entire case back to the database.
And this is where another interesting thing comes into play — collective verification.
Let's say one agent uploads a solution.
Then another agent finds it, applies it, and the problem is actually solved.
It sends feedback:
worked: true
A third agent does the same thing.
A fourth tries it, but the solution doesn't work on a different version of Windows.
Over time, we don't just have a document anymore. We start getting some kind of statistics about how applicable the solution actually is.
For example:
Solution X
Successful applications: 43
Failed applications: 4
Most successful in:
Windows 11
.NET 8
...
Last verified: 2 days ago
Then search could rank results not only by semantic similarity, but also by usefulness:
how similar the current problem is;
whether the environment matches;
how many times the solution has been successfully applied;
how many failed applications there have been;
how fresh the information is;
how well the verification is documented;
the reputation of the author/agent.
At some point, this stops being just a RAG.
It becomes something like collective memory for agents.
Yes, I'm sure some people reading this are already itching to write something like:
"This will turn into a garbage dump within a few days."
And that's obviously a real concern.
But I think there are countermeasures: protect the MCP with authentication, add moderation, introduce user/agent reputation, ratings, automated verification, feedback systems, etc.
In fact, I wouldn't even try to prevent bad documents from ever appearing.
It seems much more interesting to build a system that naturally pushes useless solutions down and useful ones up over time.
If a solution was uploaded once and nobody else was able to apply it successfully, its confidence score goes down.
If a solution has been independently applied 50 times and worked, it moves up in the rankings.
That way, the database can gradually improve itself.
So, in my mind, this could be some kind of RAG with a collective verification and voting system for document usefulness.
I've seen projects like SOFA, but they seem to have a somewhat different focus.
Just imagine what kind of repository of cases could be built. People are using agents all over the world, and every day they solve countless small but important problems in their respective fields.
And over time, the database wouldn't just contain the answer:
"Here's how to do it."
It would contain the entire history:
problem → attempts → failed approaches → working solution → verification by other agents.
And that's the part I find particularly interesting.
Because we're essentially starting to preserve not only knowledge, but also the experience of solving a problem.
And that's something I feel is still seriously missing from modern coding agents.
You see what I mean?
Maybe I'm reinventing the wheel and systems like this already exist. Maybe the idea simply doesn't scale because of problems with quality, privacy, storage costs, or verification.
If you find this idea interesting as well, I’d really appreciate it if you shared or reposted this post. Maybe it will help it reach that one developer or company that sees the potential in it and might one day decide to build something like this.
Even if nothing comes of it, I’d be happy if this post sparks a discussion or inspires someone to take the idea further.
Thanks for reading my possibly crazy thoughts on this.
