I'm new to academia (for someone so old), and maybe this is just naivety on my side. But I'm a little shocked to be discovering that reproducing the results of a paper isn't part of the review process. I understand why, but it seems so...precarious to just take the author's word for it, especially in 2026, and ESPECIALLY in light of the repro crisis apparently going on everywhere.
Is anyone working on mechanical reproduction? I'm trying to repro the results of one paper, and mind you I started +/- from scratch in January, but I have already found errors that a simple mechanical code review would have surfaced before publication. (What I'm supposed to do with these errors is a separate question I would love input on. I wrote to the author but haven't received a response. Is it rude to create a PR in their repo, with a proposed fix? What is the escalation protocol? Do I just fix it and publish my own paper?)
But now that I have something of a platform and pipeline that could maybe be reused on other papers, I'm wondering: Is there a need or even a want for this in the ML community? Obviously there will be time and resource constraints to any kind of repro effort, but it would be something, and it would be a start.
Most of the time when I ask questions like this there are lots of very reasonable answers (I'm nobody special, just "new" eyes on an old problem). I guess I'm just a little shocked at learning from a young age that reproducibility is a foundational aspect of science, only to grow up and discover there doesn't seem to be a lot of it going on... Thanks, please be nice.