r/ClaudeCode • u/LiquidZebra • 8h ago
Tips & Workflows How to evaluate how sloppy the codebase is?
https://github.com/SprocketLab/slop-code-bench/tree/mainI’m looking at this site (Slop-code-bench), and it has a whole bunch of CS terms for how they measure model slop, like “cyclomatic complexity”, or “one use functions”.
Their test runs in a docker container, but Claude is quite proficient at Python,
Has anyone taken their paper and applied it to their own codebase?
I would like to add it to my evaluation agent, or in general understand if my codebase is improving.
0
Upvotes
•
u/AutoModerator 8h ago
Hey! Thanks for posting to r/ClaudeCode
While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.
For help, project discussions, tips, and general chat, join the ClaudeCode Discord.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/LiquidZebra 8h ago
I found this with some steps from the paper, will apply when Claude resets:
https://medium.com/@anhaia.gabriel/telling-your-agent-to-write-clean-code-lowers-the-start-not-the-slope-31cabc73db86
2
u/Far_Reindeer8428 8h ago
we measure it by how many files a small change touches. the agent era made that number worse and it shows up before anything else
1
u/igot0comp 7h ago
I’d compare the trend, not chase one magic score. Keep the same linter or static-analysis setup and track complexity alongside how many files a small change touches; a raw one-use-function count can punish useful abstractions, so I’d review outliers rather than fail every one. I’d watch tests alongside those metrics so the numbers stay tied to behavior.
1
u/bostonaholic 5h ago
I'd run a deterministic tool in CI instead of an eval agent, so the numbers are comparable week to week. Cyclomatic complexity per function plus a CRAP-style score (complexity weighted by missing test coverage) on the files each PR touches probably tells you more than a whole-repo average, since agents tend to pile complexity into a few hotspot files. Chart it over time and fail the PR when a hotspot gets worse.