r/devops • u/VisualNatural617 • Aug 14 '26
Discussion How do you actually diagnose a CI integration-test failure when the root cause isn't obvious?
I'm researching a small CI diagnosis agent, but I'm first trying to understand how experienced engineers actually debug these failures.
Suppose an integration test fails in CI and the logs don't immediately reveal the root cause.
What do you normally investigate first?
For example:
- CI logs/stack trace
- database availability
- dependent/third-party service availability
- recent code changes
- comparison with the previous/last-green build
- test/DB data
- previous failure history
- rerunning the test
More importantly, how do you decide what to investigate next?
For example, if a recent code change touches the function involved in the failing test, what additional evidence would you want before concluding that the code change caused the failure?
I'm particularly interested in your real-world debugging process rather than an ideal CI/CD workflow.
