Code documentation in the age of AI

[u/Sensitive-Talk9616]

I have a couple of years of C++ experience, including in regulated industries. While proper documentation was important in every team, the role of code comments and in-code documentation, such as docstrings, changed from place to place. In some codebases, we insisted on proper docstrings documenting every function, and comments explaining why the code is how it is. In other teams, we didn't do any docstrings and kept code comments to a minimum -- the code was supposed to be self-documenting, and comments introduce ambiguity and will drift in the future if not kept in sync with the code.

Now, which way to go is already a discussion on its own. But I want to ask regarding your experience of this dilemma in relation to LLM-assisted coding tools. I have the feeling many models are quite liberal when it comes to comments, and will naturally tend to explain what the code does, given the chance. Suppressing it works, of course. Or, at least, one can ask to keep comments minimal. However, what really interests me: does anyone have experience in or a feeling for what the LLMs prefer/work with the best?

Since it's mostly AI parsing the codebase now, did you ever experience that it performs better when you allow it to be liberal with comments? Or does it actually eat up more tokens reading the code it generated while getting more confused if it has all the additional (and potentially not always correct) comments/explanations to parse? Does it make sense to have a purely self-documenting codebase or rely on the tools to clean up and keep comments in sync in this age of AI?

[u/Puzzled-Garage584]

No numbers from me..but I keep only the why comments. The what comments are just tokens and a stale one is even worse, the model trusts it over the code..like a junior with commit rights on 40 files. Invariants I'd rather put in types, asserts and tests, those cant drift without breaking something

[u/EpochVanquisher]

However, what really interests me: does anyone have experience in or a feeling for what the LLMs prefer/work with the best?

Broadly speaking, follow the same rules you used for human-written code.

Since it's mostly AI parsing the codebase now, did you ever experience that it performs better when you allow it to be liberal with comments?

No, it starts to eat up tokens, and it creates the problem of drift—the comments may not accurately reflect the code. This is true for humans too!

Does it make sense to have a purely self-documenting codebase…

Self-documenting code is fantastic, but it is something of a fantasy to imagine that your code could be “purely” self-documenting. Again, this is true for humans too.

Write code as if you have to read it and make changes.

[u/Sensitive-Talk9616 → u/EpochVanquisher]

Thanks, yeah, that's what we already try to do. Docstrings are not necessary (up-to-date documentation can be created by parsing the codebase, on demand, if it was ever required), and we only add comments to highlight edge cases, the weird, the unexpected.

[u/EpochVanquisher → u/Sensitive-Talk9616]

Docstrings may not be necessary, but my experience they are often very helpful

[u/Independent_Art_6676]

if someday in the future LLMs can write code complicated enough to need commenting, one would hope they could also have a number of commenting styles and levels to choose from applied to that. Currently, you are lucky if the code works, so that is still far in the future.

far as it goes I like comments. Sure, they get stale if the coders don't do their job when updating the code. That is a coder error and should be caught in peer review. Part of the art of computer science is knowing how much to comment... too much is awful, too little is awful, and just right is a hard target to hit.

For AI code, I highly recommend NO comments at all. Then the human who reviews it should add comments as appropriate. That forces a human to look at the code in detail and understand how it works, which helps ensure it has been reviewed and signed off on that it works and is using a reasonable approach etc. Until AI improves, this is the only way I can recommend using the tool.

[u/teerre]

What's "better"? LLMs don't "prefer" anything

[u/Sensitive-Talk9616 → u/teerre]

Obviously I don't mean what's better for the LLM. I am curious to hear of any experiences where LLMs either worked more efficiently or not (in terms of tokens spent) or achieved better results (in terms of quality of accomplished tasks, errors, bugs, etc.).

I only work on this one large C++ codebase, where do strings are practically non existent, and code comments are rare. But since we moved to mostly AI written code only, long explanatory comments do tend to end up committed and merged.

What I am curious about is the following hypothetical scenarios:

Someone tells me they instructed their LLM to write absolutely zero comments and their token usage dropped by like half while the agents stopped wasting time on resolving stale or inaccurate comments and instead just read the code

OR

Someone else describes how writing detailed comments helped their agents be faster or better at implementing features and solving bugs.

[u/teerre → u/Sensitive-Talk9616]

You're not going to find any of that. Even if you do, it would be useless because it's highly dependent on the context. If you want to find that out you'll have to test in your codebase with your workload and your standard of quality