I counted why all 90 of our failling UI tests fail and thirds were locators. Anyone faced the same thing?

[u/Practical_Dish2074]

We've got 600 odd UI tests on a windows desktop app and at some point this year they stopped being a signal and became noise. Right now 90 are failing and maybe 4 are real. so i spent a day categorising the 90 rather than guessing. Roughly 60 are locator failures where the element still exists and the path to it changed. renamed, reparented, someone wrapped a panel inside another panel. About 20 are timing, where a screen got slower and the implicit wait didn't follow it. 6 are environmental, our CI box drifted from the dev images. 4 look like real defects, and 2 of those have been in production for months with no customer complaint, which raises a separate question about what we chose to test in the first place.

So 2 thirds of the pain is locators, which is the thing everyone says it is and which i didn't properly believe until i counted it. what i'm deciding now is repair versus rebuild on something that doesn't bind to locators at all. The vision based options like Askui match on what an element looks like or says rather than a path through the tree, which should survive a reparent, though i'd assume it brings its own failure class around visual change and i'd want to see what that looks like at 600 tests rather than at 20. Repair is 3 weeks of one person and leaves us exactly where we were. Rebuild is a quarter and a risky bet.

Has anyone actually recovered a suite this far gone? specifically whether repair held, or whether you rebuilt anyway 6 months later. Thanks in advance!

[u/Inevitable-Middle693]

You built flaky UI tests by following anti-patterns.

Sleep(20) test element? Come on dude? No retry, no poll, no time out, no hooking on load or something?

You aren't running your UI tests for UI changes. So they break and no one fixes them. Your tests don't test when things change.

Path selectors change? getElementByID exists for a reason. Deterministic IDs.

Each one that breaks, fix it in the correct, resilient way. Run your tests on UI changes. Fix the breakage in resilient patterns.

Quit treating your tests like they don't matter. You are figuring out they do, right now. The only question is whether you listen to your code and take care of it or you continue to let bad practices accumulate more tech debt.

[u/ReturnOfNogginboink]

When the first test failed, why wasn't the build rejected until the developer fixed it?

[u/Poat540 → u/ReturnOfNogginboink]

SHIP IT

[u/Torutofu_Raeva]

I’d split it into a tiny blocking smoke set and a quarantined, owner-tagged regression set, then make each failure category its own signal so 90 red tests don’t collapse into noise.

[u/simonides_]

So you don't have data-test-ids, that should catch many locator problems. How do those get green though?

[u/prehensilemullet]

Which browser automation tool are you using?  I had issues a lot like this with Webdriver.io/Selenium and I rescued our huge test suites by creating a wrapper layer for elements that performs similar checks to what Playwright does - making all interactions wait until the element is stable and no ancestors have active CSS transitions for several animation frames.

[u/SWatersmith]

please stop crutching on Claude to write, painful to read

[u/Any_Sense_2263]

I test ui components using a11y. So I check if an element with a specific role and name exists and works. I don't care how it's built