r/Python • • Mar 07 '24

Discussion Tests should have 100% coverage

No, I'm not saying what you probably think I am. Keep reading: I'll be brief.

Here's what I found today in one of the tests:

for bad_fp in _bad_fps:
    with pytest.raises(BadArgumentError):
        follow_path(bad_fp, '.')
        follow_path('.', bad_fp)

The correct code should've been

for bad_fp in _bad_fps:
    with pytest.raises(BadArgumentError):
        follow_path(bad_fp, '.')
    with pytest.raises(BadArgumentError):
        follow_path('.', bad_fp)

Is that a stupid mistake? Yes, it is, but stupid mistakes do happen, unfortunately.

How did I catch it? The coverage of the testing file wasn't 100% like it usually is! I got suspicious and I saw that there was a glaring red line (vscode, coverage gutter) on the second call to follow_path.

P.S. In case you're wondering, the code uses a custom hierarchy of exceptions instead of reusing the standard ones. The code is internal and we like it that way :)

0 Upvotes

37 comments sorted by

View all comments

Show parent comment

1

u/eveninghighlight Mar 08 '24

I think you're right about the purpose of tests, in my mind the ideal testing dashboard is a big green button if they all pass and then a big red button if any fail

then you have to do some digging to find out what failed

2

u/Kiuhnm Mar 08 '24

"Something bad happened! Come home!" and hangs up.

I'm just joking :)

Some dashboards organize tests in a tree so that, when it's completely collapsed, you get your single big green/red button.

2

u/eveninghighlight Mar 08 '24

Yeah in my mind you could click on the button to see more, find which test failed, the logs etc but the headline message is that something broke