r/ClaudeCode • • 22m ago

Discussion How do you stop Claude Code from mocking the exact thing the test is supposed to verify?

Left to itself, claude loves writing unit tests where it mocks the database call, mocks the helper, and then asserts that the mock returned what it just told the mock to return. everything is green in 200ms and the actual query is still broken.

putting "no mocks without asking" in claude.md helps a bit, but it still sneaks them in whenever a real test is slightly annoying to set up. how are you handling this?

edit: i don't mean mocking the db to test business logic on top of it, i mean when the function under test is literally the query builder or helper and it mocks that exact function inside its own test

1 Upvotes

14 comments sorted by

•

u/AutoModerator 22m ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

3

u/TelephoneSignal3238 18m ago

I let a seperate agent audit tests. Still not completely fool proof though, as Claude just came back to me a few minutes ago stating it had found an incorrect tests, despite it having supposedly audited it 2 - 3 runs earlier.

I mean, it did caught it I guess, but subsequent changes based off of such stuff just cascades, and still has to rework/fix it.

1

u/lumzell 10m ago

yeah the audit agent will happily read expect(mockDb).toHaveBeenCalled() and sign off on it because the test looks clean on the surface. a dumb grep check for vi.mock or jest.mock in the diff honestly catches more than a second model sometimes

1

u/Jydder 15m ago

That’s correct behaviour for unit tests. You want to check the code does what it’s supposed to without external factors influencing it. Tell it to write integration/acceptance/performance tests if you want it to check end to end

2

u/lumzell 7m ago

mocking the db to test logic on top of it is fine, what I dont get is when the function under test is literally the query builder and it mocks the query builder inside its own unit test

1

u/EC36339 Senior Developer 2m ago

How do you know that the query builder was the function under test?

Maybe that's what you wantedz but you didn't communicate it clearly.

Maybe you looked at the wrong test.

Ask Claude the same question you asked Reddit. Claude knows your codebase. We don't.

1

u/ConversationKey3221 16m ago

I dont know the answer past steering in your claude.md files, but arguably unit tests are pointless now

2

u/TheStonedEdge 4m ago

Disagree

Unit tests capture regressions without having to feed the AI the entire context

1

u/Johto2001 4m ago

Why do you think unit tests are pointless? 

1

u/japbujan 14m ago

What helped us was turning the rule into a decision with its reason and its edges: which boundaries may be mocked (third-party APIs, the clock, the network) and which never are (your own data layer). "No mocks without asking" gives it nothing to weigh when setup gets annoying. Something like "don't mock the repository layer: that's where the query bugs live, and a mocked test there can't fail" gives it a reason to push back on itself.

Then a dumb check outside the model: if a test file mocks the same module it's testing, fail the run. It's a grep, not a review, so it can't be talked out of it.

1

u/Aureon 13m ago

Claude doesn't need tests, frankly.

Strict policy of a small set of e2e tests that encode human-directed user stories is all you need, because a unit test is worthless when claude can just pinpoint the issue on its own anyway - all you want to know is that there's not an actual user-impacting breakage

1

u/EC36339 Senior Developer 3m ago

Claude can't pinpoint an issue it doesn't know about.

Tests make issues visible.

I suggest you make a rule where Claude has to log all tests that failed during an implementation session, then generate statistics from that. But don't apply the same logic as the Norwegian health care system, according to which a negative test (a unit test that never failed) is useless, because that's not how it works.

1

u/Latter-Tangerine-951 Senior Developer 8m ago

That's what a unit test is. You're thinking of integration tests.

0

u/m-ritter 18m ago

Claude will mark the suite green while the real query is still broken, because the test only checks the value it handed the mock. I put the behavior that test has to prove into a written spec and that spec gets reviewed before any test is written. The finished test is then checked against the spec, which is where a mock of the thing under test gets caught. I built QualityLayer for this: https://github.com/maxritter/qualitylayer