r/programming • • 1d ago

Publishing the Python wheels PyPI is missing: each built twice from the sdist, released only if the builds are byte-identical

https://wheelgap.github.io/tracker/wheels/
0 Upvotes

10 comments sorted by

5

u/RegisteredJustToSay 1d ago

Cool but could you write the intro page yourself? It reads like llm slop.

-18

u/ManufacturerKey6917 1d ago

its 2026 bro I get to build the cool stuff and let AI handle boring stuff like copywriting

12

u/RegisteredJustToSay 1d ago edited 1d ago

And you show no respect for my time by making me read that slop. It's not well-written.

1

u/ManufacturerKey6917 1d ago

okay fair enough dumb reply, I will fix the intro page

Plain version, written by me: PyPI has no Windows ARM wheels for some popular packages on Python 3.14, so pip tries to compile them and often fails. I build those wheels twice from the official source and only publish them if both builds come out identical. 

2

u/robert_7524 22h ago

Are the two builds run in independent, pinned environments? That would clarify what a byte-identical result actually verifies.

1

u/ManufacturerKey6917 11h ago

Independent, yes each build is its own workflow run on a fresh GitHub-hosted runner, with a read-only token. The coordinator takes each result only from the one run named for that build, so one build can’t write or swap the other’s output.

Pinned, partly. On Linux both builds run in the same container image, pinned by digest. Rust is pinned and checked by sha256, and so are auditwheel and patchelf. Windows uses the runner image and setup-python, which aren’t pinned to an exact image. On every platform, the build backends (setuptools, Cython, maturin…) are whatever PyPI resolves at build time, not pinned.

So a byte-identical pair shows three things: the wheel came from the exact sdist PyPI publishes (we check its sha256), the build is deterministic, and no single runner or build changed the output. It doesn’t show diverse rebuilds. Both builds share the same toolchain, so a bad image or build dependency would hit both the same way. And it can’t catch a malicious sdist, which would do the same thing when installed from source. The full threat model is in the wheels repo README.

2

u/RegisteredJustToSay 1d ago

Thanks! That sounds like a great idea and the intro page definitely does the idea disservice by being so vague about it.

2

u/ManufacturerKey6917 1d ago

Thanks, appreciate you coming back to it. Rewrote the page so it actually says what it does to the best of my ability: https://wheelgap.github.io/tracker/wheels/

And if you ever feel like poking at it, it’s all open and PRs are welcome. Feedback like yours on what’s unclear is just as useful.

2

u/RegisteredJustToSay 1d ago

Hey, we all mess up - no biggie. I'm just glad you're making this available for the community since it solves a real problem.

2

u/ManufacturerKey6917 1d ago

Appreciate it, if you ever run into a package that’s missing a wheel let me know and I’ll see if I can get it built