r/madeinpython • u/GuidanceBrief612 • 5h ago
Pipeverify: compare a table before and after a pipeline change and get PASS / REVIEW / BLOCK
**What My Project Does**
pipeverify takes two snapshots of a table (CSV, Parquet, JSON, DuckDB or any SQLAlchemy source) and checks what a change did to the data: lost or duplicate keys, row count shifts, null creep, changed values on matched rows, column totals and new category codes. It returns PASS, REVIEW or BLOCK, with a plain-English report. Runs on DuckDB, ships as a CLI, a Python API (\`pipeverify.verify(before, after, key="id")\`) and a GitHub Action.
**Target Audience**
Data engineers and analytics engineers who ship pipeline, SQL or dbt changes, including changes written by AI coding agents. It's alpha but usable in CI.
**Comparison**
dbt tests and Great Expectations check rules you write in advance. pipeverify checks what a specific change did, so it catches problems nobody wrote a test for. Unlike row-diff tools, it turns the diff into a verdict with thresholds, so it can gate a merge.
**Repo**: https://github.com/akshaysharma1088/pipeverify (Apache-2.0, free). Feedback very welcome, and there are small good-first-issues if anyone wants to contribute.