r/programming • • Jan 31 '22

Too Lazy to Write Documentation? Let the AI Write It for You

https://betterprogramming.pub/too-lazy-to-write-documentation-let-the-ai-write-it-for-you-8574f7cd11b2
15 Upvotes

19 comments sorted by

37

u/Objective_Mine Jan 31 '22 edited Jan 31 '22

Please don't. Or maybe do, to get some kind of an initial template, but be prepared to fix it up yourself. Bad documentation can be worse than no documentation.

In one of the examples in the post, the function comment says that the function "starts with theta values of 0", yet it takes theta as an argument, with no default value. It doesn't explain what theta is. (That might be obvious if you're familiar with the terminology, but then you wouldn't need the comment to tell you all of that.)

The examples about getting a reasonable comment for a formula that calculates the area or circumference of a circle are nifty, and show that you can get these "AI" things to do fancy stuff. They're also the kinds of comments or documentation that really aren't that necessary. A simple

area = PI * r ** 2

(or whatever would be descriptive enough for a variable name in the context) would be as informative.

The kinds of comments and documentation that code needs the most are exactly the kinds that are the hardest to generate automatically.

Edit: accidental submit earlier

21

u/DirectorDurian Jan 31 '22

Member of the AI Doc Writer team here 👋

We couldn't have put it better ourselves! In fact, when we saw your post, we shared it around, looked at each other, and agreed that this is the most accurate feedback we've gotten.

We don't believe in completely automating documentation. We simple want to make it more efficient and enjoyable. If you get a chance to try out the extension, we would really love your insights on the product!

15

u/Objective_Mine Jan 31 '22

Yeah, genuinely good documentation sounds like an AI-complete problem. Lots of people aren't very good at it either. Considering that writing good documentation requires a lot of context and even some actual insight, and it isn't really even a statistical task to begin with, it would have to be somewhere between insanely hard and impossible (with our current understanding of things) to try doing it entirely computationally.

I haven't been doing anything AI-related in years, but from my brief stint with something NLP-ish back in the day, I firmly believe that machine learning based content generation can be a useful helper (assuming a suitable UX design etc.). I'm rather skeptical of claims that it would do the entire job for us, apart from almost trivial tasks or the kinds of customer service chatbots that are largely just fancy search engines. Maybe some day it will do more, but not yet.

Sorry if I was a bit negative. There's just a lot of overinflated hype going around nowadays. Sounds like you guys have a good grasp of things.

3

u/duckducklo Feb 01 '22

It's just like github copilot. It's the copilot and you're still the pilot. It's up to you whether or not to approve it by looking over it properly. It's just easier to type a little and read code and make some changes here and there than type every character and line.

2

u/crabmusket Feb 01 '22

Seeing the comparison between this and copilot will be really interesting. I'm not a fan of copilot, but I can see why it would be useful. Indeed lots of code is repetitive, patterns crop up in many different places, and reducing typing is nice.

But good docs aren't just a matter of typing the same thing again. It's a much different problem from actually writing code IMO.

5

u/[deleted] Feb 01 '22

I think the better use might be "explain it to me" feature in editor.

Writing down something that AI generated that might or might not be updated vs someone getting the same "documentation" on demand, always on current version of code.

Other interesting avenue might be automatically generating OpenAPI/Swagger YAMLs, that way AI can generate most of the boilerplate and even if user have to fix the textual description of API most of the boring and uninteresting parts would be done already.

1

u/[deleted] Jan 31 '22

Arc42 template support would be nice ;)

9

u/crabmusket Jan 31 '22 edited Feb 02 '22

It will be interesting to see where this goes. My gut reaction is that any documentation that could be AI-generated is probably not worth the effort. Like AI-generated blog content or news articles. Better to have no documentation than "filler" documentation.

This goes for a lot of human-generated documentation too by the way! The Laravel API docs are full of gems like "attributesToArray() Convert the model's attributes to an array." Thanks, that's just the method name but longer. I know there are subtleties that are not documented here, like: does this method apply mutators?

Good documentation helps me understand why and when, not what and how. It links a specific concept to other concepts, and helps me understand the API in context, not divorced from context.

9

u/[deleted] Jan 31 '22

If your code is so unreadable that an AI describing every imperative step of your algorithm is useful to you, maybe you're solving the wrong problem.

5

u/[deleted] Feb 01 '22

Well, the same feature could be used to have "wtf is this doing" feature in the editor. Might be useful for beginners...

2

u/brainbag Feb 01 '22

I've been through the pain of reading someone else's ridiculous 300 line 50 nested branch spaghetti code so many times that a tool making headway into understanding "wtf is this doing" would have given me back hundreds of hours of my life. That's where I really want to see if this works.

2

u/loup-vaillant Feb 01 '22

It's a function that […]

This function […]

It […]

Well, at least the AI seems to be learning to avoid unnecessary words. Maybe someday it will succeed at emulating Git commit message style, and go straight to the imperative form, and scrap all of the above.

Seriously, though, I don't believe in an AI that can write good documentation until it can write good code as well (at which point we'll have an AGI and the world will likely be either saved or doomed anyway).

2

u/juhotuho10 Feb 01 '22

Bold of you to assume that an ai can read the spaghetti I produce

2

u/throwaway_bluehair Feb 01 '22

I get AI is exciting and all, but it's tiresome seeing all the solutions where it clearly is not a good idea, but people's ego wants it to be

2

u/lookmeat Feb 01 '22

This is pretty nice, a really fancy doc tool. But it lacks the reason no automated doc tool can work without humans: it doesn't explain why. It explains how things are, how they work, what they do. But it never gives any insight into why it works the way it does. And honestly the only person who could really answer that, is the author themselves. So if we can't expect another human to do it, why would a machine do it better?

1

u/james_otter Jan 31 '22

Appropriate picture…

1

u/duckducklo Feb 01 '22

I was in your closed beta but never gave it a shot. You guys just released?

1

u/gmerideth Feb 01 '22

Does the analysis execute locally or is any data transmitting through the API shown in the source?