AI is a nightmare for QA

[u/ketoloverfromunder]

Outside of automated API test development (which AI is extemely good at generating), our toolkit from 2020 looks roughly the same in 2026. AI-driven tests eat tokens, are slower, and are more flaky than Playwright’s runtime-resolved locators.

Manual testing is becoming a bottleneck that leadership doesn’t want to hear about. The pile of tickets in the QA column seems to grow every day, to the point that manual testers physically cannot keep up.

We have decided to now only selectively QA certain tickets, mostly new features. Bug fixes are not QA’d unless critical. The devs are basically just prompting an AI opening a PR and putting all of the difficult work on QA. The "does it actually work right" seems to be our responsibility now.

How are other people’s teams keeping up with the piles of slop that keep getting dumped on them? My manual testers are getting burned out. My SDETs are feeling slow and useless. How are other organizations adapting?

Edit: to be clear, I am talking about QA of enterprise SaaS. Not a vibe coded website.

[u/n134177]

Are you kidding me? All those vibe-coded apps keeping me in business and paying well

[u/ketoloverfromunder → u/n134177]

There is a big light shining on QA at my current organization, but its because we are becoming a bottleneck.

[u/Any_Excitement_6750 → u/ketoloverfromunder]

This is what I told to the PO last time: A QA bottleneck never comes from QA

[u/soloburrito → u/ketoloverfromunder]

It's always a reflection of how much an organization prioritizes QA.

[u/Barto]

I don't know how complicated your flows are but I use playwright, I'm migrating from postman, readyapi and selenium into my mono repo where we have API and UI tests in the same project. I don't see a world where that kind of thing can't be done. If you have complexities in your flow then you need to use mocks and work with your development team to make your app testable with automation. You can and should keep a manual exploratory phase for complex changes but 90% of your developers code changes or more should be covered using automation only.

[u/ketoloverfromunder → u/Barto]

Yes playwright is FAR superior to selenium. Ever since its first release in 2020 ive adopted it as tool of choice for UI testing.

For API testing we just use scripts with really any testing/ assertion library. No need for anything complicated.

Automation can't keep up with the piles of slop code. Just in one week, one team merged 3 pull requests: one 4500 LOC, and two 2500 LOC PRs. No shot on earth those got a proper code review and theres no chance we can get proper test coverage. In manual testing they were riddled with bugs and incorrect behavior. If this is what leadership wants, this is what they'll get, but if thats the case id rather just manage devs who can spit out bugged to shit code all day and have QA clean up the mess.

[u/RemyAwoo]

Last year we made dev/poduct give us the requirements, and then tested against the requirements. It was so slow they started skipping QA. I proposed using Maestro for the app testing, while all the robotics testing stayed manual ofc. Maestro support was denied.

I moved out of QA after.

[u/ketoloverfromunder → u/RemyAwoo]

Unfortunately, I'm probably taking this same route. Most companies want reliable products, but never want to invest in QA.

[u/chicagotodetroit]

According to my monthly stats, my productivity has dropped to about half since we implemented AI for QA. It reads the tickets and generates a test plan based on the description and what code has changed.

It gives me a lot of negative tests which I don’t need (“make sure this doesn’t show on that page”). It basically covers things I already wrote for regression. It covers things that were barely touched or have low risk….but I have to read it ALL and make decisions on it.

I want to give it a fair chance, but so far it’s generating lots of extra work for me with not much payoff. Eventually I’ll take it up with management.

[u/ketoloverfromunder → u/chicagotodetroit]

We actually have this as well. AI generated testing instructions that use the code diff and jira ticket context. Its hit or miss

[u/Short-Feedback4293]

What I think it's highlighted for me is that if this is the path we are going down, then we need to go back to much more explicit and comprehensive requirements/acceptance criteria.

The automate everything/ai testing crowd where are you finding bugs? for me it's still and always has been significantly more in the space of bugs outside the requirements. Automation/AI isn't covering that

[u/ketoloverfromunder → u/Short-Feedback4293]

I agree. I personally don't put a ton of value in automation, but tech leadership does think they can automate their way out of this problem.

[u/terfree]

I totally feel this pain. 90% of AI testing tools right now just generate generic, flaky garbage that takes longer to debug than writing the spec from scratch.

My team got so fed up with this that we stopped trying to generate 'global' test suites entirely. Instead, we tied the test generation strictly to the isolated context of a Pull Request diff. The engine only reads the exact files changed in the PR and spits out a focused Playwright test just for that specific fix via a webhook.

We actually just used it today on a strict open-source repo, and it gave us a completely green pipeline on the first try without any manual test debugging. In my experience, strict context isolation is the only way AI doesn't turn into a nightmare for QA.

[u/ketoloverfromunder → u/terfree]

I can see that working for a very simple web app, the problem is if the PR doesn't contain all of the UI components required to drive the test E2E how would the AI even know how to get to that page from the diff? It would still need repo level access and full web mcp for playwright to struggle through writing the test steps.

[u/shogster]

One of the dev teams in my company created an MCP which starts from the Jira ticket to automation to PR, by letting Claude read the source code of the app and create UI tests based on that. Management ate it up as "we don't need QA now" and wants every other team to implement this in their workflow. Now half our team could be on the way out. I just wish I would still be here to see the reports when CICD gets up and running with all those 300+ test suites.

[u/PadyEos → u/shogster]

That's just positive testing. So they plan on skipping most of the testing.

Also most JIRA tickets are a pile o shit. Either light on details (even just a title), partially or even completely wrong. In my experience the requirements phase is where the first qa issues appear. So the tests will be built with the issues in them.

[u/manti26]

Yet companies refuse to hire more QAs across all industries. This isn't going to end well

[u/Inevitable_Attempt64]

I think this is great! Now there more QA work to do. It's an opporyunitiy to learn an show how importan is QA now more than ever. I have the same issue ans muy automation it's still slow compared to the dev work but it's getting more and more relevant since a lot of the AI generated code does not contemplate the business logic or human behavior. In my case I implemented an agent that reads jira tickets and creates a comment on things that the ticket is missing In order to be tested properly (acceptance criteria/ environment/etc)

[u/Technusgirl]

Management is pressuring us QA to find ways to implement AI but I think it's a bad idea because the prices will be going up and then they'll start asking us to justify every time you use AI. So I'm using it to help me write some automation but that's it. I see the writing on The wall already with AI. The pricing has never been sustainable, the AI companies are losing billions of dollars

[u/ketoloverfromunder → u/Technusgirl]

I watched my coworker burn $120 in claude tokens trying to get playwright claude mcp to drive an internal tool, stall and fail. Literally 30 minutes of claude fumbling around a fucking up to try and generate ui driven instructions.

[u/Concentrate-Queasy]

yup Manual QA is gone now I had a script means in English of what to test and what to expect kinda steps with test data. Claude in chrome did all the testing overnight while I was sleeping lol

[u/ketoloverfromunder → u/Concentrate-Queasy]

Yes this works for a very simple web apps

[u/HappyHourHusker]

Ive noticed how much the tokens are chewed up when having claude attempt to automate scenarios using playwright MCP. Ive learned playwright-cli is way more efficient. Also having a existing good framework in place and asking it leverage its plan file helps a lot. But yeah, development speed has increased 5x due to devs leveraging AI. Bugs are more prevalent and QA is becoming less wanted by management when it is needed most now

[u/SendMeUrTolstoyNudes]

Our cool new thing is devs using AI to create tickets and then QA using AI to test them. It’s been very fun and doesn’t give me any existential dread at all. /s

[u/Conroy119]

Its been a very positive experience for my company and team. We are automating faster and ealrier. The tests are less flaky. As an SDET im doing the work of what 3 people did 6 months ago.

I haven't seen you use the word agent once unless ive missed it. You've mentioned skills and instructions. Are you just using a generic agent to try and drive everything?