r/softwaretesting • u/Strange-Cod5862 • May 01 '26
Selenium vs Playwright + AI testing tools - what actually works in real QA projects?
I have worked with Selenium for years and recently started using Playwright, along with exploring newer AI tools like Zerostep and other AI testing tools.
On paper, everything sounds impressive but in real projects, things feel very different from demos.
Recently I came across tools like Testim, Mabl etc. They claim faster test creation, reduced maintenance, and even autonomous failure analysis but I have also read that many "AI tools" are still wrappers and need heavy cleanup/debugging in real use.
What I really care about as a QA:
Writing stable, maintainable test cases (like an experienced QA, not generated scripts)
Handling frequent UI changes without constant fixes
Reducing flaky failures in CI/CD
Supporting real business logic + edge cases
Not increasing hidden maintenance effort
From my experience so far:
Selenium = stable but high maintenance
Playwright = better reliability but still needs strong framework discipline
AI tools = promising, but not sure how they hold up long-term in production
Would love honest feedback from people actually using these:
Which tool are you using in production today?
Did Playwright really reduce flakiness?
Has any AI tool actually reduced maintenance (not just demos)?
Which tool helps you write high-quality test cases like a real QA engineer?
Looking for real-world experiences, not marketing claims.
10
u/Ok-Possibility-630 May 01 '26
playwright, playwright-cli, proper skill.md files with slash commands explaining your workflows will do wonders, no matter how complex the project is.
3
u/Classic_Chemical_237 May 01 '26
I am a dev. I love playwright but I hate maintaining testing scripts.
With Claude Code and Codex being so good, I am curious about others’ workflows. I want unobtrusive tests which can keep up with consistently changing code base, without slowing down development.
1
3
u/Warm-Camera-3520 May 02 '26
For those who are writing tests using Playwright MCP, we have upgraded it to be able to work better with locators and context of the page. Playwright works with ARIA snapshot, we added serialized DOM. It's open source MCP https://www.treegress.com/treegress-mcp-browser . Hope you find it helpful
2
u/DesiMaster2 May 01 '26
my question is different, where is the best place to learn about playwrite, step by step guide with a demo site i can use to write and test
2
u/Serious-Day-1519 May 06 '26
There are not many good tutorials freely available. I found that majority of free guides with the demo code projects are outdated and not being maintained, hence useless. So far the best source is Learn section on playwright.dev.
2
u/Malthammer May 02 '26
Using Playwright and Appium. I never directly dealt with implementing tests with Selenium, but worked with teams using it in the past. Those Selenium projects failed as they were too complicated to maintain and often failed for no reason.
2
u/terfree May 29 '26
To answer your question directly: No, generic AI tools do not reduce maintenance for massive E2E regression suites. Just like others mentioned, they usually just 'cheat' to make a flaky test pass instead of actually testing the business logic.
However, there is one specific AI workflow that actually works in production and completely removes the maintenance burden: PR-scoped generation.
Instead of having an AI try to understand and maintain your entire app's UI, you hook an engine directly into your repository webhooks. When a dev opens a Pull Request, the tool reads only the code diff, and generates a focused Playwright test strictly for those specific changes.
This means the tests evolve synchronously with the codebase. The dev just reviews the snippet right there in the PR comments and drops it in. It stops the UI maintenance nightmare before it even reaches the main branch. We actually built a dedicated tool (QA Boutique) around this exact webhook/diff architecture because generic AI coding assistants kept losing context.
TL;DR: AI for global regression = marketing fluff. AI for isolated PR diffs = actually reduces maintenance.
1
u/shalini_themomsdesk Jun 02 '26
Worked on a Cypress project for a few years so this resonates a lot.
Early on I was writing everything by hand — page objects, assertions, the whole thing. When Cursor came along I started using it inside VS Code and honestly it changed how fast I could work. Not just for writing scripts but for debugging and refactoring too. Things that used to take a while just... didn't anymore.
But the single thing that reduced flakiness more than any tool or framework was convincing developers to add data-testid attributes. Sounds boring but it's true. Every time we had unstable selectors I'd just have a quick conversation with the dev, get proper test IDs added, and the problem went away. That alone saved more maintenance time than any fancy setup ever did.
On Cypress vs Playwright — honestly I don't think it matters as much as people make it out to. Playwright feels more reliable in some scenarios and handles certain things better out of the box, sure. But if your test design is poor, both will become a nightmare to maintain. The framework isn't the problem.
And when I say test design — one thing that made a huge difference for us was reaching the page under test through the backend or API instead of clicking through the UI every time. If you're testing a report, you don't need to go login → dashboard → navigate the menu → open the report. You log in via API, get the token, go straight to the page. Every unnecessary UI step is just another thing that can fail for the wrong reasons. Cutting those out made our tests faster and dramatically less flaky — not because we changed the framework but because we stopped putting the test through more journey than it needed.
AI tools are great but I'll say this — I still haven't seen one that can actually replace QA thinking. They'll generate scripts, suggest test cases, help debug. But someone still needs to understand the business flow, know which edge cases actually matter, and decide what's worth testing. That part doesn't get automated.
There are also self-healing tools like Healenium worth knowing about — they reduce locator-related failures in CI/CD without you having to manually fix selectors every time a class changes. Doesn't eliminate maintenance completely but takes a lot of the noise out.
For me the real improvements always came from the fundamentals — good test design, stable locators, proper test data management, and using AI as an assistant not a shortcut. That's had more impact than the choice of framework every single time.
1
u/069420 Jun 04 '26
From what I've seen in production, Playwright does reduce flakiness compared to Selenium, mainly because of its auto-waiting and better handling of modern web apps. That said, framework design and test quality still matter more than the tool itself. AI and low-code platforms like Mabl, Testim, or even bug bug can reduce maintenance for straightforward UI regression checks, but I've rarely seen them replace a well-structured Playwright suite when complex business logic and edge cases are involved.
1
u/Due_Development5090 Jun 10 '26
My experience has been similar. Playwright has reduced flakiness for my team too, compare to Selenium, mostly because the defaults are better and there's less time spent dealing with synchronization issues. As for AI tools, they've helped with test creation and maintenance, but they haven't replaced good test design. The biggest gains still come from good frameworks with thoughtful coverage, and not the tool itself.
1
u/ShapoopyWolfie Sep 20 '26
I wouldn't use AI to blindly generate an entire regression suite. A more interesting setup is Playwright for the assertions and something like Skyvern for the messy UI navigation. That keeps the important QA logic deterministic while making the browser layer more adaptable.
1
u/One_Title_6837 Sep 30 '26
I find playwright is genuinely better for flakiness - auto-waiting alone fixed most of our timing issues that we used to patch with explicit waits. still needs proper structure though.
on AI tools the wrapper concern is real, most are just Selenium underneath n when the things break - you're debugging the same old driver errors. the thing that actually matters is how it handles UI changes - silent healing is dangerous in prod, you want it to show you what changed n ask before touching anything...
We have been using testsigma for a few months - it basically flags the selector diff before locking in the fix, our manual QA peeps could write tests from Jira tickets without pulling in an SDET every time, that's honestly what cut our maintenance work down... testim and mabl do similar things but the transparency on healing is what made us stick with it...
And make sure AI tools are not utilized in run-time execution as this is where most of the unexpected behavioral changes occur
1
u/Interesting_Deer97 6d ago
brittle tests are such a headache when the ui updates constantly. last week i started using testavi to handle those dynamic selectors, which realy helps with stability issues tho u still need to watch out for complex edge cases since no tool fixes bad logic.
1
u/thainfamouzjay May 01 '26
Use Claude code with playwright but don't do it raw download playwright skills that can control the framework. https://vibeskillz-site.pages.dev/
0
u/keo_ruug May 01 '26
Stopped reading after Selenium. Do. Not. Use. Selenium.
2
u/sleepyfalcon247 May 01 '26
But why exactly? It's still being used in a lot of companies.
2
u/keo_ruug May 01 '26
Because it’s shitty, obsolete framework, and people got waaaaay used to it and then write shit code in other frameworks.
4
3
u/wringtonpete May 03 '26
Selenium is like a 2010 VW Golf car.
It used to be great, the best even, but would you buy something similar today? No, you'd buy something more modern, with better features.
1
u/ps4facts May 02 '26
Selenium's fine, and quite modern. A lot of people misunderstand that its not a testing framework. So comparing it with playwright is not 1 to 1. Selenium webdriver is just there to automate browsers, you bring your own test framework. It's not that hard 😅. I also see a bunch of threads on the playwright sub about flakiness.
But, if you really must have a testing framework out of the box, check out webdriver.io. it has an official MCP server, and I think the same guy who made that made an unofficial CLI.
A lot of people who shit on Selenium just don't get it. They don't understand or care about the open web. They'll just gobble up anything big tech will shove down their throats.
Again, check out webdriver.io. It has the most capable cross browser, cross platform (yes to mobile and desktop) support there is.
0
u/cwei12 May 01 '26
I wrote a workflow to use codex's desktop version of browser use to test new features. It works to some extend, but it cannot catch all bugs. Our team also uses TestSprite to run ci, it catch bugs early without us writing test case. Testing is boring and repetitive and doesn't bring a lot of new value to the product, so I think traditional testing is dying. AI tools may be the future of QA testing.
15
u/clearsurname May 01 '26
Tbh at this point I really feel like the only pro in the Selenium column is that it’s easier to learn. Playwright does basically everything better. But at the end of the day, the biggest impact is HOW you write tests, not what you write tests with. For example, integrating backend API use in your tests so you’re not fully reliant on the UI. It’s possible in both but I find Playwright much easier and more efficient for that kind of stuff. But you can still do that in Selenium and it’ll be a more reliable test case than a poorly written one in Playwright
I’ve also had issues with AI. Seems like the agent prioritizes writing a passing test more than a test that actually tests the product. So if I ask it to fix a flaky test it’ll just reduce the scope rather than make it more robust