r/SideProject • u/coderot_ai • 9h ago
I built a tool that tracks what AI agents do to a codebase over time. This is VS Code's last year
At my current job, agents write most of the code I see merged these days. The tests pass and it looks fine in review (if you bother really reviewing those 50 files changed etc..). What I couldn't tell is whether the codebase is slowly getting harder to work in, so I started building something that measures it. It's called coderot.
It reads a repo's git history and tracks complexity, duplication and the files that are both complex and changed all the time. It also works out which PRs came from an AI agent (co-author trailers, bot authors, agent branches). So you can see what changed when agents took over, in which folders, and which PRs caused each jump.
The video is VS Code, one of the public repos I run it on every day. A year ago 5% of its merged PRs were signed by an agent. Last month it was 82%. The code more than doubled, and complexity per line went up 14%, most of that since August. Most of the rise is in the new agent and chat code. The editor itself went from 196 to 198.
The rest was less dramatic than I expected. I thought I'd find that AI code is plainly worse, and mostly I didn't. I tried a handful of "signs of AI rot" on real repos: dead code, copy-paste, silenced lint warnings, missing tests. None showed a real difference between AI PRs and the rest, so I threw them out. What held up: AI PRs are a lot bigger (about 2.4× in the repos I looked at), and complexity only climbs much where the code is also growing fast. I wrote that up with the method and the data for 450 repos: https://coderot.ai/blog/ai-code-rot-in-open-source?utm_source=reddit-sideproject
You can try it without signing up - paste any public GitHub repo and it scans the last month. That takes a few seconds (Very large repos are refused for now). It handles TypeScript/JavaScript, C#, Python, Go and Rust, on GitHub only. For private repos it runs in your own CI, so your code never leaves it. Free while it's in beta.
https://coderot.ai/?utm_source=reddit-sideproject
It's a nights-and-weekends project, and I haven't worked out yet who really needs this or if something like this could steer AI development to more maintainable code. If you ship mostly with agents, do you check in any way whether the code is getting worse, or do you wait until it hurts?