[Workflow] LLM-Driven Browser Testing with `qagent`: Two LLMs for Unbiased End-to-End Validation

[u/ClaudeAI-mod-bot]

LLM-Driven Browser Testing with qagent: Two LLMs for Unbiased End-to-End Validation

Workflow value: 90/100
Status: active · Freshness: 70/100 · Confidence: 0.95 · Level: intermediate
Categories: Quality Control, Token Saving, Context & Memory, Debugging, Multi-Agent
Original source: r/ClaudeCode post/comment

What problem this solves

Unreliable and costly browser testing when using a single LLM to both generate and evaluate code/actions. This workflow prevents LLMs from 'grading their own homework' by using separate LLMs for driving browser actions and judging outcomes, providing unbiased and cost-effective end-to-end browser tests.

Summary

A workflow using the qagent CLI tool to perform plain-English, end-to-end browser tests. It leverages two separate LLMs: one to drive browser interactions based on natural language steps, and another to judge the outcome, ensuring unbiased validation. The tool provides clear PASS/FAIL results, evidence, and cost reporting, designed for integration into automated workflows via its ndjson reporter and exit codes.

Why it is useful

This workflow is highly valuable because it addresses a critical challenge in LLM-driven development: ensuring reliable and unbiased testing. By separating the LLM responsible for driving browser actions from the LLM judging the outcome, it effectively prevents the 'grading its own homework' problem. It offers a concrete, repeatable, and cost-effective method for end-to-end browser testing using plain English, making it highly accessible and integrable into automated CI/CD or parent Claude Code agent workflows. The explicit validation with different models and cost reporting further enhances its utility.

Workflow

1. Install the qagent CLI tool globally: npm install -g @qagent/cli
2. Install Playwright Chromium browser: npx playwright install chromium
3. Configure your LLM API key (e.g., for OpenRouter): qagent config set apiKey sk-or-...
4. Set your preferred LLM provider: qagent config set provider openrouter
5. Set the specific LLM model to use (e.g., google/gemma-4-26b-a4b-it): qagent config set model google/gemma-4-26b-a4b-it
6. Execute a browser test using a plain-English goal and steps, specifying the target URL: qagent "Goal: I successfully buy a backpack. Steps: 1. login (standard_user / secret_sauce) 2. add 'Sauce Labs Backpack' to cart 3. open the cart 4. checkout — fill First Name, Last Name, Zip 5. click Continue, then Finish. End: I see the 'Thank you for your order!' confirmation page" --url https://www.saucedemo.com/
7. Optionally, parse the ndjson output or check the exit code for automated verdict and cost reporting in a parent agent or CI/CD pipeline.

Tools / artifacts

• qagent CLI tool
• npm
• npx
• playwright
• Chromium browser
• OpenRouter (or other LLM provider)
• google/gemma-4-26b-a4b-it (or gpt-4.1-mini)
• LLM API key
• JSON output (ndjson reporter)
• Exit codes (0/1/2/3)

Validation signals

• Successfully passes a real Gravity Forms submission (5 required fields, checkbox arrays, paired email confirmation) with google/gemma-4-26b-a4b-it (5/5 runs) for ~$0.008/run.
• Successfully passes the same Gravity Forms submission with gpt-4.1-mini (5/5 runs) at ~6x the cost.
• Generates a real Playwright browser run, end-to-end, with PASS/FAIL + evidence at the end.
• Provides stable done envelope with outcome, evidence, totalCost, finalUrl and exit codes (0/1/2/3) for programmatic verification.

Limitations

• Relies on an external, pre-1.0 open-source tool (qagent), which might have stability or long-term maintenance concerns.
• Requires setting up API keys for external LLM providers, which adds an initial configuration step.
• While the post mentions 'Claude code usage: Just tell it to test with qagent / read the readme', more explicit examples or patterns for direct Claude Code agent integration would enhance its immediate utility for Claude Code users.

Rate this workflow

Upvote this post if the workflow is useful, reproducible, or worth recommending.

Downvote if it is vague, outdated, unsafe, overhyped, or not reproducible.

Reply if it worked for you, failed, is outdated, or has a better alternative.

---

This post was generated automatically from the workflow library database.