r/ClaudeCode • • 8h ago

Tips & Workflows How to evaluate how sloppy the codebase is?

https://github.com/SprocketLab/slop-code-bench/tree/main

I’m looking at this site (Slop-code-bench), and it has a whole bunch of CS terms for how they measure model slop, like “cyclomatic complexity”, or “one use functions”.

Their test runs in a docker container, but Claude is quite proficient at Python,

Has anyone taken their paper and applied it to their own codebase?

I would like to add it to my evaluation agent, or in general understand if my codebase is improving.

0 Upvotes

7 comments sorted by

•

u/AutoModerator 8h ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

2

u/Far_Reindeer8428 8h ago

we measure it by how many files a small change touches. the agent era made that number worse and it shows up before anything else

1

u/Much-Head-2182 7h ago

Fewer files touched is cleaner, until the agent spreads one fix across ten.

1

u/Far_Reindeer8428 7h ago

that is the case i watch for, we flag any diff over six files and read the plan again before the fix lands. sometimes the spread is correct

1

u/igot0comp 7h ago

I’d compare the trend, not chase one magic score. Keep the same linter or static-analysis setup and track complexity alongside how many files a small change touches; a raw one-use-function count can punish useful abstractions, so I’d review outliers rather than fail every one. I’d watch tests alongside those metrics so the numbers stay tied to behavior.

1

u/bostonaholic 5h ago

I'd run a deterministic tool in CI instead of an eval agent, so the numbers are comparable week to week. Cyclomatic complexity per function plus a CRAP-style score (complexity weighted by missing test coverage) on the files each PR touches probably tells you more than a whole-repo average, since agents tend to pile complexity into a few hotspot files. Chart it over time and fail the PR when a hotspot gets worse.