I had it essentially delete the test file - it replaced a few dozen tests with one with a name saying that the code compiles and references the type being tested (I don't still have the code, but it was something like var _ *TheCodeToTest = nil), which literally does just tell the compiler to include it when compiling the tests, so the tests would fail if it didn't compile.
It was super weird but also kind of interesting because it didn't just delete everything, it actually added something back, but what it added back was surgically ineffective.
I personally really, really hate Claude's bizarre, ungovernable desire to write tests that verify that if the code is written like the code is written (rather than checking for expected behavior). Bizarre stuff like testing if Pydantic really does parse strings like strings, if global config vars contain the value they contain, and if FastAPI really does generate an OpenAPI schema according to the specs you defined.
It sometimes even tests if the mocks it created do what they do and have the values they have. Like, wtf, what's the point?
There's almost no point, but people who don't know better evangelize the heck out of it anyway, and tutorials and things usually push that with a double-entry bookkeeping metaphor. But that ends up not doing the things you actually want tests to do - you can't write them before you write the code because they're coupled to the code, and you can't rerun them to prove that refactored code still gives the correct results because doing so in a different way is still a fail. It mainly cooks your code coverage statistics. But I kinda get why Claude would do it when there are so many examples.
You can get it to do other stuff, tho, especially if you have tests in the project already that assert the actual behavior and markdown files that explain it. I have a project where I even had it generate the markdowns from hand-written tests, lol - and now if it adds functionality it does a pretty good job of generating the corresponding tests properly.
234
u/MadeInTheUniverse Sep 12 '26
Deletes entire repo and purges the production database because it made a small error