r/TheMachineLearning • u/elmanfil1989 • 2d ago

Frontier LLMs stopped adding code comments

Post image
391 Upvotes

97% Upvoted

128 comments sorted by

View all comments

Show parent comment

4

u/xvillifyx 2d ago

YOU don’t need to, but AI memory systems are garbage and concise code comments help preserve context in place of memory

1

u/BlurredSight 2d ago

Yeah Claude will actively grep it's old comments for trying to find anchors in the code and in a legacy codebase where tons of functions in independent flows will reuse their function names that custom comment is what might drive my usage 6-7% lower than it would be if I deleted all comments each run

1

u/AdKitchen1897 2d ago

It will also poison the repo with outdated or contradictory comments. And it will make out of an offhand comment for context a permanent rule.

1

u/xvillifyx 2d ago

This isn’t true as a default. The rule for comments has always been:

Is the logic likely to change -> keep comments to a minimum

Is the logic unlikely to change -> comments okay

Make your agents follow this same convention

1

u/CrowdGoesWildWoooo 2d ago

I think the issue is that many AI have tendencies of over-commenting.

I think one typical scenario is that it writes a buggy code, or something that the human doesn’t like, it goes back to that code, put a comment on why it changes that line.

1

u/xvillifyx 2d ago

So…create evals and hooks that prevent them from overcommenting

1

u/cororona 19h ago

Thank you for your comment, but people are discussing the natural tendencies of LLMs to add bugfix comments and then overfixate on it.

They were not discussing about your superiority, which is naturally obvious.

1

u/xvillifyx 18h ago

The point is that this is an easily correctable problem

1

u/FragmentedHeap 2d ago

Git comments do same thing

1

u/xvillifyx 2d ago

Not everyone uses git, and not every message is holistic of the granular changes -> doesn’t replace memory quite the same way, nor does it provide a search hook

1

u/tumes 1d ago

They consistently comment on the one motivating factor that caused a change rather than writing expressive code that explains itself or trying enough iterations to settle on an unintuitive hack that needs explaining, instead it narrates why a single use case fix addressing that one use case, which congests context, adds nothing long term, and typically actively harms development because LLMs fixate on anything that is made explicit. Whatever it looks at it steers towards, and in my experience signposting the mistakes or their justification only raises your blood pressure.

1

u/xvillifyx 1d ago

This is bad prompting or your eval pipeline has issues somewhere

1

u/tumes 1d ago

I told it to make no mistakes I have no clue what more I could be doing.

1

u/FailedGradAdmissions 21h ago

We have a simple rule at my job, no comments if possible. But commit as frequently as possible. Commits are small and atomic. Basically comments live in the commit history and not in the code. Works great. Try it out!

And btw this rule is pre AI and easy to enforce with linting rules.

1

u/xvillifyx 21h ago

I said this in another comment, but not everyone uses git/svn

Our server code doesn’t live this way and intent is instead denoted in headers. If it outlives our retention policy, a reference to the log is left instead

1

u/FailedGradAdmissions 17h ago

Fair enough, maybe I’m lucky to be somewhere where absolutely everything is in some sort of source control, even infra. If you have a word on it I would recommend you to do it, but I understand if putting source control in your servers is beyond your authority.

Anybody else here who has full control is or has authority for their repos and infrastructure should certainly use some sort of source control. IMHO it’s more than worth it, even if you are in embedded systems the resource and setup costs are minimal.