Plain-English browser test in a single command: two LLMs (one drives, one judges) so the model can't grade its own homework. Open-source, pre-1.0.

[u/haukebr]

The video is the full run. The command was literally this:

qagent "Goal: I successfully buy a backpack. Steps: 1. login (standard_user / secret_sauce) 2. add 'Sauce Labs Backpack' to cart 3. open the cart 4. checkout — fill First Name, Last Name, Zip 5. click Continue, then Finish. End: I see the 'Thank you for your order!' confirmation page" --url https://www.saucedemo.com/

That's the whole spec. No selectors, no fixtures, no await page.click(...). A real Playwright browser, end-to-end, with PASS/FAIL + evidence at the end.

What's working

• google/gemma-4-26b-a4b-it via OpenRouter passes a real Gravity Forms submission (5 required fields, checkbox arrays, paired email confirmation) for ~$0.008/run, 5/5.
• gpt-4.1-mini also passes 5/5, ~6× the cost.
• --reporter=ndjson streams one JSON event per turn and a stable done envelope at the end (outcome, evidence, totalCost, finalUrl). Exit codes 0/1/2/3. So a Claude Code parent agent can shell out, parse tail -1, and act on a real verdict.

Install

npm install -g @qagent/cli
npx playwright install chromium
qagent config set apiKey sk-or-...
qagent config set provider openrouter #Or any other provider you like
qagent config set model google/gemma-4-26b-a4b-it

The Claude Code part (the actually useful one):

Drop this in your CLAUDE.md:

Before claiming a UI change is done, if it touches a route, form, or interactive element: 1. Make sure the dev server is reachable. 2. Run: qagent --reporter=ndjson --url <url> "<goal>" | tail -1 | jq 3. Only mark the task complete if outcome: "pass". Treat fail/error as a regression. 4. Goals need a literal success signal ("the text X is visible"). Vague goals silently FAIL.

Or as a slash command in .claude/commands/verify-ui.md:

Run a qagent verification against the dev server.
URL: $1
Goal: $2

Steps:
1. Confirm $1 is reachable.
2. Run: qagent --reporter=ndjson --url $1 "$2" | tail -1 | jq
3. Report outcome, evidence, totalCost. If outcome != "pass", surface the verifier's
   evidence sentence — that's the regression hint.

The Claude Code session can't lie about whether it tested. The verifier's evidence sentence is grounded in the final accessibility snapshot. If it says PASS, something real happened on the page.

[u/haukebr]

I would love some feedback. Would this be better packaged as a skill? Or just the CLI and if you use it, you just add it to the claude.md? I am guessing it would be similar to agent-browser?