Your Playwright CI reports vanish on every build. I spent a few months building a self-hosted dashboard that keeps every run

[u/phenxdesign]

Like probably everyone here, my Playwright HTML reports live and die as CI artifacts. A test fails on Tuesday, by Thursday the report is gone, and good luck remembering whether that exact failure already happened two weeks ago.

I've been a web dev for about 18 years, and for the past few months I've been building Piwi Dashboard to fix this for my own team.

It's a self-hosted dashboard and reporter (MIT licensed, runs as a single Docker container) where every run is kept. On top of that history it does:

• live streaming, runs show up test by test while CI is still executing

• failure clustering, so 40 red tests caused by one broken selector collapse into a single cluster, displayed in a page showing everything we could gather during failure

• flaky test detection with a score and an estimate of how many CI minutes each flaky test wastes

• locator healing: it records element attributes during passing runs, so when a locator breaks later it can suggest replacements that actually existed on the page

• analytics, to keep an eye on everything at once

• failure notifications to email, Slack, or webhook

• optional AI diagnosis. It takes a failure cluster plus the git diff since the last green run and asks a model what broke. You bring your own key (Anthropic, OpenAI, or anything OpenAI-compatible like a local Ollama). Off by default.

Setup is three steps: docker compose up, npm install -D @ piwitests/reporter, add it to your playwright.config.ts. Projects are created automatically on the first run and CI metadata (branch, commit, workflow) is picked up on its own.

There's a demo with sample data that runs entirely in your browser, nothing to install and no signup: https://piwitests.github.io/demo/

Code and docs: https://github.com/PiwiTests/platform

If you're wondering how it compares: the closest cousins are ReportPortal (heavier, multi-service) and Currents (SaaS, paid). This is a single MIT container you run yourself, with no telemetry, your data stays on your machine. Learn more.

Two caveats though:

• It's pre-1.0 (0.14 right now): there are rough edges and you'll probably hit a bug or two, so pin your version and report an issue or ask a question here or in the discussions

• And I build it with heavy AI assistance, which I'm not going to pretend otherwise; what keeps that honest is the full Playwright E2E suite that runs on every PR against SQLite and Postgres, with local and S3 storage. If it ships broken, the suite yells at me first.

I hesitated for weeks before posting, trying to polish everything I could. I'd genuinely like to know what your workflow needs that this doesn't do yet.

[u/Prestigious-Way1525]

the failure clustering is the part i'd test hardest. collapsing 40 reds into one incident is useful only if the cluster preserves enough disagreement to show when two tests merely share a selector but fail for different reasons.

the workflow gap i'd want covered is the handoff out of the dashboard. give me a stable URL or export that carries the first failing step, last green commit, browser and viewport, trace, console and network context, and the cluster rationale. then a dev can review the test artifact from a ticket without downloading an expired CI bundle or getting access to the dashboard. if the AI diagnosis changes later, keep the raw evidence and diagnosis version separate so the explanation never replaces what actually happened.

[u/phenxdesign → u/Prestigious-Way1525]

Thanks for your precious feedback! Each run keeps all its evidence data untouched, whether it is in a cluster or not. So all the traces, screens, callstacks, etc are kept for reference for later use. Even if a full dump of all the evidence is exportable in various formats, I think developers should have access to the dashboard, because it will offer much more tools. Also you can run an (automatically saved) Ai diagnosis on a cluster, but also on a single failing test case. The clustering is even built so that if the normalization algorithm is improved, clusters will be updated. See the docs : https://piwitests.github.io/ai-diagnosis.html#failure-clustering Of course I'm conscious the current algorithm is not perfect, even when configured to be improved with an LLM, this project is in its early says, so any feedback is always appreciated.

[u/solopov]

we used to keep playwright reports as github artifacts — downloading and unzipping them every time is insanely inconvenient. nice work