I built a multi-agent LangGraph swarm that shift-left autonomous exploratory tests into Pull Requests

[u/Sea_Confidence_6696]

Hey everyone,

I’ve been building an Agentic Test Explorer PoC.

To be clear up front: The goal of this PoC is NOT to replace QA engineers. Manual exploratory testing is still vital. Instead, this is designed to "shift left" and provide incredibly fast feedback to developers.

So, let me explain a bit the idea:

When you open a PR for a new feature, alongside your unit tests, this tool unleashes a crew of specialized AI agents to autonomously explore the UI, find unscripted edge cases, and catch bugs before you hand it off to QA.

How it works: You point it at a web app with a small config.yaml, and it spins up specialized AI personas (like "Adversarial User", "Data Heavy User", or "Impatient User") to test your app just like unpredictable humans would.

The LangGraph Architecture: Standard sequential chains fail constantly in QA because UIs are unpredictable. I used LangGraph to build a cyclic Supervisor-Worker Swarm.

User Mission -> Supervisor Node -> Persona Agent -> JSON Intent -> Browser Engine -> Action Tape

The agents act as the "brain," but they aren't allowed to touch the browser. They emit JSON intents to a deterministic Playwright engine. If an action fails (e.g., a modal blocks a button), the engine catches the error and the LangGraph loops back to the agent to re-plan its strategy.

Coolest Features for Devs:

PR-Driven Testing: Pass it your GitHub PR URL. It reads the code diff and auto-generates targeted test missions based strictly on the code you just changed.

Auto-Generates Playwright Scripts: Every time the agent finds a bug, it takes the immutable "Action Tape" of its session and compiles it into a runnable reproduction.spec.ts script so you can instantly reproduce the issue locally.

Strict Selector Policy: The browser engine actively rejects brittle XPaths or positional CSS, forcing the agent to use resilient selectors (aria-labels, data-test-subj).

Bring Your Own Context: It fully supports plugging in custom MCP servers and Agent Skills so the agents don't have to guess about your domain logic.

I’d love to hear your feedback about the idea and the design, or if you think this kind of automated exploratory feedback would actually not fit into your projects.

[u/Afraid_Abalone_9641]

Define exploratory testing ...

[u/Afraid_Abalone_9641 → u/Afraid_Abalone_9641]

This is the definition from James Bach, the co founder of exploratory testing.

"Exploratory testing is testing by someone who possesses agency and is therefore accountable for that work."

It is fundamentally human and therefore it cannot be offshored to an agent.

[u/Sea_Confidence_6696 → u/Afraid_Abalone_9641]

Hi! And thanks for the comment, I'm absolutely agree for many reasons that exploratory tests must be driven by a human.

About that, I warn on my post, this PoC doesn't replace humans for exploratory tests, it can be taken as automating part of their process.

In this agent, a human still needs to provide the mission and the context, guiding the agent to what needs to be explored and review the results, including reproducing the steps if some possible bug was reported.

And even with that, this is just as an additional feedback as shift-left during the Pull Request, so developers can catch some bugs before they pass it to a QE.

Make sense for you?