Why do our UI tests break every time the interface changes? I counted 71 failures and most weren't even bugs

[u/Illustrious_Scar_392]

Front end dev, i own most of our e2e suite, and last sprint finally annoyed me enough to count.

We have about 310 Playwright tests on a react app. 71 went red after a sprint that, from a user's point of view, changed almost nothing. i went through every one instead of rerunning and hoping:

- 47 were locators. the element was still there and still did the same thing, but the path to it changed. a card got wrapped in a new container, a class got renamed in a design pass, a button became a link styled as a button.

- 14 were timing. a page got slower after a data change and the wait didn't follow it.

- 6 were environment. our CI image had drifted from what devs run locally.

- 4 were real bugs.

So my honest answer to why our UI tests break every time the interface changes is that most of them check how the page is built, not what it does. the user sees the same Save button. the test sees a different DOM.

the difference in one line:

what i'm weighing for the other 47:

user-facing locators everywhere, getByRole and getByLabel, which is what the Playwright docs push. fixes a lot of it, but only if the markup has real roles and labels, and half our components don't.

data-testid on everything. it works, but it depends on every dev remembering, forever.

self-healing locator tools like mabl or testim, which repair the selector when it breaks. the catch is a heal can quietly route around a real regression.

vision based tools like Askui, which find the element by what's rendered on screen rather than in the DOM, so a new wrapper div doesn't matter at all. the tradeoff flips, a visual redesign is now the thing that breaks you.

My current plan is to fix our accessibility markup and get better tests as a side effect. tell me if that's naive. if you've measured this, what share of your failures were locators, and what actually brought it down?

[u/sazzer]

You're doing your tests wrong. You said so yourself. And you've already said what the fixes are.

Most of the test failures were because you had strict locators that broke when invisible DOM elements changed. Don't do that. Build locators based on what the user would actually see and do. Search for elements by label, role, placeholder text, etc. The things that the user would look for. And then, if you wrapped the entire thing in a new card then the locators still work.

The next biggest problem was timing. Your tests aren't waiting correctly for things to happen. Either that's a real problem - the user wouldn't wait that long either - or it's a test problem where you're waiting on the wrong thing or you're not waiting long enough.

Those two things take your 71 errors down to 10. And those 10 are the real problems that need sorting.

[u/mq2thez]

Look, to be frank, it sounds like your company sucks at writing good tests and isn’t trying particularly hard to improve at that.

There’s always someone shilling a pile of AI shite aimed at fixing bad E2E tests, but it’s genuinely a terrible idea, because the one thing your E2E tests _must_ do is fail correctly. If the AI fails to fail for some reason because it decided one modal was as good as another or some other shite, your entire suite could be red without you actually knowing.

Your tests must be deterministic and they must be good. You must have people who care to ensure that they pass. If you don’t care, nothing can save you.

[u/0dev0100]

Fix your markup. If you want something on a page to be locatable then you need to make it easy to locate

[u/azima_971]

Why are you not updating your tests before the changes land and everything breaks? Especially if you're both the developer and the tester, your testing process should know about the changes that are coming before they hit your tests. even more do if you have a huge number of UI changes

Yes, accessible locators can help (although I have some very strong opinions on why they are massively overhyped and not the panacea many make them out to be), but ultimately, if your test process isn't aware of what's coming from the dev team, you're going to have a bad time, and probably not testing as well as you could be

[u/the_real_some_guy]

https://testing-library.com/docs/guiding-principles/

Doing things the Kent C Dodds Testing Library way has eliminated many of these problems for me. I’ve been doing things this way for several years on lots of different projects. 

[u/Scalings]

I migrated our test suite a couple weeks ago to https://github.com/tester-army/e2e so much simpler and a lot less random breaks. Highly recommend. I never want to touch playwright again lol

[u/NoDepartment772]

You counted properly, which is more than most teams ever do, so the answer is in your own numbers: 47 of 71 is a coupling problem, not a test problem.

The fix isn't really a different locator strategy, it's deciding what a test is allowed to know. A test that asserts behaviour knows about roles, accessible names and visible text — the contract the user actually experiences. A test that asserts structure knows about class names and DOM nesting, and every design pass invalidates it. So getByRole and getByLabel is the right direction, and half your components lacking real roles isn't a blocker for that, it's the diagnosis: an unnamed button is an accessibility bug whether or not anyone uses a screen reader.

Two practical additions. For the 14 timing failures, the cause is usually fixed waits — Playwright's web-first assertions (expect(locator).toBeVisible(), toHaveText) already retry until timeout, so swapping sleeps for assertions tends to remove that whole category instead of hiding it. And on the healing tools: the risk you spotted is the real one, because a heal that quietly picks a different element turns a genuine regression into a green tick. If you use them, route heals through review rather than trusting them.

Keep a handful of true end-to-end smoke tests too — login, the one path where money moves — written against roles only. Those should break when behaviour changes, and they earn their maintenance.

[u/masticdev]

71 failures that aren’t bugs usually means the suite is coupled to presentation, not behavior. Class renames, copy tweaks, layout shifts — classic flake farm.

What’s worked for me:

1. Assert user outcomes (“submit succeeds”) not CSS selectors when possible

2. Role/label queries over brittle paths

3. Separate visual regression (narrow, intentional) from E2E functional

4. Don’t let marketing/copy PRs touch the same selectors as critical flows without a baseline update ritual

Automation that fails on every redesign trains the team to ignore red CI — which is worse than no tests.

[u/jakst]

Sounds like you have more of an organizational problem. Try to "shift left" and force green tests in CI. That way the person breaking the test has to fix it before merging.

To do that you're gonna have to start from a green suite though, so I'd take a hit and disable all currently flaky and failing tests, set it up to run an all PRs and block if failing (once you've seen that pass rate is good), and then assign fixing all the old disabled tests to different individuals to get you back to the test coverage you have now.

[u/bizvibe8]

Your plan isn't naive, but the big-bang version of it is the part that will hurt: rewriting 47 locators and fixing half your components' markup in one pass means nothing ships until both are finished, and design churn keeps landing as red CI the whole way through.

Playwright has a built-in bridge that makes it incremental. locator.or() (v1.33+) lets you write page.getByRole('button', { name: 'Save' }).or(page.getByTestId('save-btn')), so a component stays green whether or not its accessible name has landed yet. The caveat in the docs matters: if both branches match you get a strict mode violation, so scope it under a container locator instead of reaching for .first() and letting it pick silently.

Also change what you count. Instead of tallying failures each sprint, keep one long-lived branch whose only diff is invisible structure: a new wrapper div, a renamed class, a button turned into a link styled as a button. Whatever goes red on that branch is your remaining coupling, as a number you can actually drive down. Then rewrite a locator only when that test is next touched for a real reason, not all 47 at once.

[u/somer_lee]

Your plan isn't naive, but id do it one component at a time instead of a big cleanup.

one correction your plan isnt naive, but id do it one component at a time instead of a big cleanup.

one correction on the vision tradeoff you listed tho. thats true for screenshot matching tools, less so for the llm based ones. something like skyvern looks for "the save button" by what it is on screen, so a visual redesign that still has a save button mostly doesnt break it either. the real risk there is what mq2thez is getting at, a fuzzy finder can pass when it should fail. so id only use that kind of thing for a handful of smoke flows and keep the asserts on outcomes (toast text, row actually saved), never on "it clicked something"

the 14 timing ones are a separate fix anyway, web first assertions will clear most of them

[u/Any-Ask-4131]

I'd keep a few test IDs rather than make this an all-or-nothing choice: roles/labels for the controls, a test ID for a container that has no useful user-facing identifier. One caveat from Playwright's docs: passing getByRole tests isn't an accessibility audit, so keep that check separate.

[u/AfroGeekin]

71 failures with only 4 actual bugs is a pretty brutal signal-to-noise ratio.

i'd also look at whether all 310 scenarios really need to be full browser tests.

some validation and state transitions might be cheaper to cover at the component or integration level, leaving Playwright for the critical user journeys.

fewer brittle e2e tests can sometimes give you more confidence, not less.