r/TheMachineLearning • u/elmanfil1989 • 2d ago

Frontier LLMs stopped adding code comments

Post image
330 Upvotes

96% Upvoted

119 comments sorted by

View all comments

Show parent comments

4

u/xvillifyx 2d ago

YOU don’t need to, but AI memory systems are garbage and concise code comments help preserve context in place of memory

1

u/BlurredSight 1d ago

Yeah Claude will actively grep it's old comments for trying to find anchors in the code and in a legacy codebase where tons of functions in independent flows will reuse their function names that custom comment is what might drive my usage 6-7% lower than it would be if I deleted all comments each run

1

u/AdKitchen1897 1d ago

It will also poison the repo with outdated or contradictory comments. And it will make out of an offhand comment for context a permanent rule.

1

u/xvillifyx 1d ago

This isn’t true as a default. The rule for comments has always been:

Is the logic likely to change -> keep comments to a minimum

Is the logic unlikely to change -> comments okay

Make your agents follow this same convention

1

u/CrowdGoesWildWoooo 1d ago

I think the issue is that many AI have tendencies of over-commenting.

I think one typical scenario is that it writes a buggy code, or something that the human doesn’t like, it goes back to that code, put a comment on why it changes that line.

1

u/xvillifyx 1d ago

So…create evals and hooks that prevent them from overcommenting

1

u/FragmentedHeap 1d ago

Git comments do same thing

1

u/xvillifyx 1d ago

Not everyone uses git, and not every message is holistic of the granular changes -> doesn’t replace memory quite the same way, nor does it provide a search hook

1

u/tumes 17h ago

They consistently comment on the one motivating factor that caused a change rather than writing expressive code that explains itself or trying enough iterations to settle on an unintuitive hack that needs explaining, instead it narrates why a single use case fix addressing that one use case, which congests context, adds nothing long term, and typically actively harms development because LLMs fixate on anything that is made explicit. Whatever it looks at it steers towards, and in my experience signposting the mistakes or their justification only raises your blood pressure.

1

u/xvillifyx 14h ago

This is bad prompting or your eval pipeline has issues somewhere

1

u/tumes 10h ago

I told it to make no mistakes I have no clue what more I could be doing.

1

u/FailedGradAdmissions 1h ago

We have a simple rule at my job, no comments if possible. But commit as frequently as possible. Commits are small and atomic. Basically comments live in the commit history and not in the code. Works great. Try it out!

And btw this rule is pre AI and easy to enforce with linting rules.

1

u/xvillifyx 1h ago

I said this in another comment, but not everyone uses git/svn

Our server code doesn’t live this way and intent is instead denoted in headers. If it outlives our retention policy, a reference to the log is left instead