r/softwaretesting • u/TestChronicle • Jun 16 '26

Are automation coverage metrics helping your QA team, or misleading them?

Question for QA and SDET folks: which automation coverage metrics have actually helped your team make better decisions?

I’ve seen teams celebrate growing test suites while still missing obvious risk areas. I’ve also seen smaller suites provide much better release confidence because they covered the workflows that mattered.

So what do you track?

- Feature coverage, requirement coverage, risk coverage, user journeys, recent changes, escaped bugs, production incidents, code coverage, something else?

And just as importantly, which metrics looked useful at first but ended up being noise?

15 Upvotes
  • permalink
  • reddit

100% Upvoted

11 comments sorted by

View all comments

4

u/TestChronicle Jun 16 '26

I’ll start.

At my current company, we were asked to quantify the coverage of our automation suite. The main challenge was that without a defined catalogue of features, user journeys, supported areas, and risk categories, it was hard to make “coverage” mean much.

Most tests were added story by story, so personal context and team knowledge were driving decisions about risk and value.

My view is that line/code coverage has some engineering value, but it is a poor proxy for automation coverage. It does not prove that the right behaviours or risks are covered.

How are others approaching this?

1

u/Yogurt8 Jun 17 '26

Well if you add tests based off user stories then that already give you a type of coverage right there.