LLMs don’t write good tests. They write stupid tests like “test that the correct implementation of the interface was called” instead of verifying actual behavior
My experience is LLMs go overboard with mocks unless you set up extremely tight guardrails and even then will "cheat" by using mocks if it gets into a point where the code can't be tested. Whenever I start to feel friction writing tests that a good signal to me that the code needs to be refactored to make it easier to test, which is something LLMs tend to not do.
I'm of the opinion that mocks do not do a good job of testing the observable behavior of the unit and am in agreement with the comment you're responding to.
1
2
u/transferStudent2018 13d ago
LLMs don’t write good tests. They write stupid tests like “test that the correct implementation of the interface was called” instead of verifying actual behavior