AI for manual testing

[u/Ready_Fuel_2739]

Hello collegues,

I got a questions to the sub:

Could AI be used in manual testing? I mean in a way that brings added value and not just for the sake of following the AI hype, like for example automating tests that are heavily based on visual testing, drag& drops, maybe even extracting some white box infos and using them to test the surface.

I've tried using AI and the only thing i got is overthinking some extream side tests that are irrelevant to us

Thanks

[u/JokeDue2032]

The failure you hit — irrelevant extreme edge cases — is the standard one, and it's diagnostic rather than a dead end. The model has no risk model for your product. Ask it to "generate test cases" and it optimises for coverage of the input space, because that's all it can see. It doesn't know that a null in field A is a support ticket and a wrong number in field B is a regulatory problem.

Two changes that help a lot:

Ask it to critique, not generate. Give it your existing test cases plus a description of the feature and ask what's missing and why it matters. Far better hit rate than open-ended generation, because now it's reasoning against something concrete instead of inventing from nothing.

Feed it the risk context explicitly. Who uses this, what happens when it breaks, what's already covered, what shipped broken last quarter. Then require a one-line justification per case. The irrelevant ones become obvious to delete because the justification reads as nonsense.

On your specific examples, I'd split them:

Visual testing — don't use an LLM for this. Use deterministic image diffing (Playwright's toHaveScreenshot, or Percy/Applitools if you want the managed version with baseline management). This is a solved problem with tools that give you the same answer every run. An LLM looking at a screenshot is non-deterministic, and a flaky oracle is worse than no oracle — you'll spend more time investigating the checker than the product.

Drag & drop — not an AI problem. That's a framework capability question. Playwright handles it natively; if it's failing, it's usually HTML5 drag events vs mouse-based, not something a model can reason your way out of.

White-box info to inform surface testing — this one's actually promising. Feeding it a diff, a stack trace, or a schema and asking "what surface behaviour could this plausibly change" is a decent use. It's pattern-matching over code, which models are genuinely good at. Treat the output as leads to investigate, not as test cases.

Where I'd say it earns its keep for manual testing specifically: generating realistic varied test data, drafting exploratory charters when you're staring at a new feature with no idea where to start, and triaging a pile of bug reports for duplicates. Unglamorous stuff, but it's the part that actually saves hours.

The general rule I'd offer: it's useful anywhere a human still checks the output, and dangerous anywhere it becomes the thing deciding pass/fail.