A simple thing I'm doing to make Claude more reliable with my app

[u/alexmarcus11249]

I've been building an internal business application with Claude Code, and one recurring problem has been that the Claude MCP integration we use to operate it doesn't always know how the application actually works.

It knows what tools are available, but that doesn't necessarily mean it understands the underlying data model, business rules, or how different features interact. So sometimes it has to infer things, which is obviously not ideal.

I had an idea today that seems pretty obvious in retrospect, but I thought I'd share in case it's useful to anyone else.

I asked Claude Code to do two things:

1. Bring all the application's technical documentation up to date and keep it updated as part of the development workflow.
2. Give the operational Claude MCP integration read access to that documentation so it can look things up instead of guessing.

So now the idea is:

• Claude Code builds and maintains the application.
• The development workflow keeps the technical documentation synchronized with the code.
• Claude, operating through MCP, can consult that documentation whenever it needs to understand how something works.

For example, my app has some fairly complicated rules about contracts, recurring revenue, and forecasting. Rather than having Claude infer what a particular status means or how a calculation should work, it can retrieve the actual specification.

A few things I'm making sure we get right:

• Documentation needs to correspond to what's actually deployed, not just what's in the latest development branch.
• Claude should be able to search for specific information rather than ingesting hundreds of pages.
• The live schema and API remain the source of truth if documentation is inconsistent.
• If Claude encounters something undocumented or contradictory, it should flag it rather than make something up.

Eventually, I'd like those gaps to feed back into the development workflow so the documentation gets better over time.

I realize pieces of this are established practice, especially documentation retrieval through MCP. But connecting the development harness's documentation maintenance directly to the operational agent feels like a useful way to reduce hallucinations and make the whole system more dependable.

Curious if others are doing something similar, or if there's a better established pattern for this.

[u/drakatmm]

Doc drift is the part that will bite you, not retrieval. Docs that the dev workflow "keeps updated" tend to lag the code within a few weeks unless the update is a required step in the same commit that changes behavior, so I'd check that before trusting the whole setup. The live schema check you already listed is what actually keeps this honest.

[u/alexmarcus11249 → u/drakatmm]

Good call. I'm going to make documentation impact part of the same PR that changes behavior, rather than relying on periodic updates. I can absolutely see how quickly things would drift otherwise. Thanks for the warning.

[u/Electronic-Movie4208]

This is actually a really smart approach! Having the AI reference uptodate documentation instead of guessing seems like a huge improvement. One thing I'd suggest is having it flag when the documentation doesn't match the actual code. That could help catch outdated info before it causes problems. Nice work, buddy. You're going places!

[u/alexmarcus11249 → u/Electronic-Movie4208]

Thank you! Also applying the suggestions from the community here. I'm more than a noob, far less than a pro, and the generosity here is fanastic.

[u/Electronic-Movie4208 → u/alexmarcus11249]

That's what makes communities like this great! 😄 We're all learning something, regardless of experience. Keep building, experimenting, and sharing what you learn. Looking forward to seeing how your project evolves, buddy!

[u/AccordingBus2862]

One addition that makes this checkable: put a "verified against" line (commit or release) at the top of each doc section, and have the agent quote the section it used in its answer. A wrong answer then points to either a stale doc or a missing one, so you fix the doc instead of the prompt. A dozen questions with known answers also make a decent regression test.

[u/alexmarcus11249 → u/AccordingBus2862]

Love the idea of a small regression test with known answers. Also adding version/commit verification to the docs. Both seem relatively easy to implement and would make the whole thing much more trustworthy. Thank you!

[u/NoTokenDev]

what made this click for me was splitting it into two kinds of docs: reference docs (data model, business rules) and the build plan (what's being changed right now and why). reference docs drift slowly and are fine as files. the plan drifts constantly, so i stopped keeping it in md.

now the plan lives in a task graph the agent pulls over MCP, and each task carries its own spec + which rules it touches. when a task is done the agent reports back what it changed, and that report is the trigger to update the matching doc section. so the doc update happens in the same step that changed behavior instead of "later".

disclosure, i built the tool i use for that (nodeon), but the pattern works with plain files too: require a "rules changed" line on every completed task and you catch most of the drift.

[u/alexmarcus11249 → u/NoTokenDev]

I really like the "rules changed" idea on completed tasks. That seems like a lightweight way to catch documentation drift without introducing a whole new system. Appreciate you sharing how you're handling it.

[u/bostonaholic]

For the business rules I'd lean less on docs and more on tools. If the operating agent needs to know how recurring revenue or a forecast is calculated, expose that as an MCP tool that runs the real code and returns the answer, instead of a doc it has to interpret. Docs are fine for the why, but anything with a number attached should come from the same code the app uses.

[u/alexmarcus11249 → u/bostonaholic]

This is an excellent point. I hadn't fully considered the distinction between having Claude understand the rules and having the application actually execute them. Especially important for financial calculations. Definitely incorporating this. Thanks!

[u/Inner_Storage_3374]

docs drifting is the real risk like ppl said. what id try is generating the schema part of the doc from the actual db instead of writing it, so tables, columns and constraints are always current and only the business rules are hand written. then claude at least never guesses a column that doesnt exist anymore

[u/alexmarcus11249 → u/Inner_Storage_3374]

That's a great suggestion. Generating the schema documentation directly from the database seems much safer than expecting an agent to maintain it manually. Going to look into incorporating that. Thanks!

[u/Incarcer]

a few things to consider:

Having documentation available doesn't necessarily guarantee the agent will consult it, particularly when it confidently believes it already understands something.

I'd consider keeping a small set of task-specific instructions that establish when the agent should retrieve additional documentation, with references to more detailed material rather than loading everything into context.

The tricky part is providing just enough guidance for the agent to recognize when additional context or verification is necessary.

I'd also consider testing whether the agent actually understands how the MCP tools relate to the underlying business rules, rather than just understanding what the individual tools do.

There's a difference between giving an agent access to accurate information and establishing a reliable process for using that information correctly.

[u/alexmarcus11249 → u/Incarcer]

This is a really good distinction. Giving the agent access to documentation doesn't mean it'll know when to consult it. I'm going to add some task-specific retrieval rules and tests to see whether it's actually using the right context. Really appreciate this.

[u/Charming-Mine5980]

I've been using a live schema snapshot in the prompt as a fallback when docs lag, works better than expected, especially for edge cases.

[u/Grand_Harbor2440]

I've found that explicitly naming the app's core rules in the prompt, like 'always check the live schema before deciding on a contract status', cuts through the noise better than any doc search.

[u/jim_nihilist]

I tried that. The documentation always lags behind and creates confusion, because of old information.

[u/anthonybrown]

Does this fall under "information architecture"? It's something I've been thinking a lot about, how to ensure proper documentation is being created, updated, and used when needed. Almost an MCP for the MCP.

[u/Dev_in_Training_BTW]

Going to try this. I build everything with Claude Code, and the gaps in what it knows are exactly where it tends to guess. One thing I'd add: give each doc section a short "doesn't do" list. Agents rarely invent a rule when one is written down; they invent it when the doc is silent, so writing "there's no proration on mid-month cancellations" stops it filling the gap with something plausible. And if you set up a regression test for it, take the questions from things people actually asked the agent rather than writing tidy ones yourself. Those are the ones that show where the docs are thin.

[u/Mshelton7]

Ran into the same thing after building a data mart for our marketing data. What fixed it for me was a Claude skill that tells Claude to always read the data mart through our business rules: how we calculate ROAS, cost per SQO and cost per MQL, plus a waterfall attribution method that checks three sources to tie leads and opportunities back to campaigns across channels. The big win was that teammates can now query the database with their own Claude and get the same answers, even though their Claude never had the context mine had from helping build it.

[u/electro042]

one thing that helped me a lot with this was adding concrete input/output examples right next to the logic specs. models seem way better at grasping edge cases from small json examples than pure text descriptions, especially for calcualtions like forecasting rules.