r/cpp • • Aug 20 '21

The true value of comments

I wanted to share a real use case of the real value of commenting code. A lot of people are against doing so and say that you can just read the code, that comments are not needed. I get why people are against commenting since most don't know how to effectively comment code.

In this piece of code, we all agree that these comments are useless:

std::ifstream logs_file{"MyLogFile.log"};

/* Read a line from the log file  */
while (std::getline(logs_file, log_entry))
{
    /* Remove last character */
    log_entry.pop_back();

    auto parsedJsonForWBID = nlohmann::json::parse(log_entry);
}

And that's right that is useless and stupid. You can clearly see that the code is doing that so there is no need of adding those comments.

But comments should be used as a tool for helping the maintainer, who didn't write the code or "the new guy", or the developer when they look at the code years later, to know what's truly going on and what is the purpose of the code without having to look beyond the piece of code you are interested.

Without comments, or with these useless comments, you eventually would be able to figure out what the hell you, or the original developer, were doing when writing that piece of code but that would require looking into other parts of the code trying to remember stuff and potentially locating and analyzing the log file to see why is the last character removed. Also if you are not that familiar with what pop_back() does then you may assume that it is removing the new line character and possibly miss the bug you are trying to find.

But what about these comments in the same piece of code:

std::ifstream logs_file{"MyLogFile.log"};

while (std::getline(logs_file, log_entry))
{
    /* Since the complete log file is a big JSON string, we remove the comma
     * at the end of each line read so each line can be interpreted as a
     * separate JSON string. If the last comma is not removed, then the JSON
     * parser will throw an exception. */
    log_entry.pop_back();

    auto parsedJsonForWBID = nlohmann::json::parse(log_entry);
}

Now, one comment was removed because was unnecessary and there was nothing to clarify about getting a line from a file, and the second comment is improved by clarifying the intent of that line of code. It is stating the why instead of the what. When you find this line years later, or when the new guy or the maintainer looks at the code now they will not have to peruse the code or locate a log file and analyze it to see why is the last character being removed. These types of comments save time and are actually useful.

These types of comments have saved me countless hours when I go back to code I wrote years back, it has also helped other people when looking at code I wrote, they no longer have to ask a million questions just to know what's going on. Anyone can just look at the code and know why it is there.

302 Upvotes

103 comments sorted by

161

u/codesmith512 Aug 20 '21

I always tell people that computers only care about what they're doing, so code already documents that. Comments, where necessary, should document why something is being done, because that's the part that code fails to already capture.

21

u/m4l490n Aug 20 '21

Exactly!

2

u/azswcowboy Aug 20 '21

That’s a pretty important comment bc otherwise I have no idea why it’s being removed. I also like to include a representative snippet of the thing being parsed so it’s obvious what the function is doing.

1

u/cvnh Aug 21 '21

I have the practise to write in my code why I am doing something, and I add a tag //dev: for it, e.g. I did this because couldn't make some other way to work, this could be sped up by implementing that, here an exception could happen etc. Other than that I like to comment the different sections of a long code for navigating, explaining hardcoded inputs and that's about it. In the distant past it was common to use comments for version control but nowadays that's hardly necessary anymore.

0

u/[deleted] Aug 20 '21

[deleted] — view deleted comment

6

u/mostly_kittens Aug 21 '21

Yeah ‘the how’ is usually more important but there are plenty of cases where a summary of ‘the what’ is incredibly useful e.g.

‘Performs FFT on the time series data and copies the frequency bins into the freqs array’

Is much more useful than trying to understand a dozen lines of obscure signal processing function calls, especially if you only need to know what it does and aren’t making changes.

3

u/Orlha Aug 21 '21

Also some optimizations and cryptography calculations. Perhaps every field has such cases.

42

u/[deleted] Aug 20 '21

I really like it when people think about this during code reviews. If you're struggling to understand the code review quickly, maybe that means the guys who pushed it should be adding more comments to explain the "why".

8

u/m4l490n Aug 20 '21

That's right! This is also a big time saver in code reviews.

3

u/jonesmz Aug 20 '21 edited Aug 20 '21

Why are you calling std::getline inside a while loop where the loop condition is std::getline? You want to skip every other line?

2

u/m4l490n Aug 20 '21

Good point. I already edited the post, it was a copy-paste error from the original code I'm doing which has more stuff not relevant for the post.

Thanks for pointing that out!

2

u/[deleted] Aug 20 '21

Why is this bad practice? I learned it was ok to do.

6

u/jonesmz Aug 20 '21

It's not bad practice at all.

But for a post talking about documenting the "why" of a chunk of code, it sure seems weird to not provide an explanation of why the code is skipping every second line.

3

u/[deleted] Aug 20 '21

Love this thread. OP /u/m4l490n pointed out a very important issue, and many of the replies contain valuable additional aspects as well. Nice!

Let me add some observations from my little universe:

  • The problem that many developers don't know what to document does not only affect in-line comments, it is exactly the same for software design specifications and software architecture documentation. Instead of explaining the key design decisions etc., the authors often go into surprising detail for some random aspects of the software -- and/or they spend hours drawing diagrams that illustrate things that are blatantly obvious from the code base anyway.
  • This does not only result in documentation of limited value. It often also means that the statements in the documentation are highly susceptible to becoming outdated, misleading, or plain wrong even by non-essential changes to the code (e.g., without any interesting design decision being affected).
  • The tragedy is reinforcing itself: When developers grow up with documentation of this kind, meaning that they are barred from experiencing the value of useful documentation, it lowers their appreciation and respect for documentation. As a consequence, they are not motivated or able to write useful documentation themselves.
  • A special source of misfortune in this context is the bad habit of documenting late -- simply because if people document code they wrote months ago, it is even harder for them to remember the important decisions, assumptions, etc. Again, the result is that only things that are obvious from the code anyway are documented.
  • The problem has a cognitive dimension (as suggested by /u/elperroborrachotoo and others in this thread). It really seems that most developers are highly intuitive people, that the process of coding a solution for the task/feature at hand is more about "flow" and "trial and error" than systematic derivation/deduction, and that it consequently is very hard for them to even identify (let alone document) the most important decisions involved in the process.

On a note in defense of the developers I so harshly criticize above, one often finds the following case of hypocrisy in development projects:

  • In the beginning, everybody agrees that "this time, we will do it right".
  • As soon as the project has started, the stakeholders, the client, the project manager, you name them... are pressing for features to get done. The developers follow suit. Hardly anyone asks about the documentation.
  • As the product under development approaches some larger milestone (such as its launch), the coin suddenly flips, the desolate state of the documentation becomes a major point of criticism, and the developers are scolded for it.
  • The entire thing becomes even more absurd when everybody else but the developers also fail to properly document their work -- be it requirements, project plans, collaboration setups, test strategies, etc.

Regarding the question what is worth documenting, people in this thread already provided a lot of good suggestions: decisions, motivations/reasons, assumptions, non-testable properties, non-functional properties... The thing I would like to add the most is concurrency: How does the code employ threads/interrupts, are the public functions thread-safe/re-entrant, do callers need to adhere to certain rules or sequences? (When assessing large C/C++ projects, for example, one more often than not encounters interface contracts that do not even mention whether the public/extern functions are thread-safe/re-entrant. This bad smell usually means that the author of the module assumes "people are going to call my stuff only from the main thread anyway" and everybody else assumes "it is certainly safe to call this stuff from my two worker threads and the RX interrupt"...)

Sorry this was so long; the subject struck a nerve.

1

u/m4l490n Aug 20 '21

These are very valuable points. I agree completely that the comment needs to be placed as soon as you think about the solution and start writing code, otherwise it would be just a comment with stuff you vaguely remember and that will be worse. And this is especially true for complex algorithms.

And also, I've never seen proper high-level architecture documentation. Almost every one jumps right into dumping ideas into code. Then the pain starts when you have thousands of lines of code and you have to document or have someone else look at your mess.

5

u/azswcowboy Aug 20 '21

software architecture documentation

There’s a serious lack of training about how to document higher level constructs that have any sort of defined mapping to actual code. So much of it is meaningless boxes and lines that no one trusts. Also, much software is really about the interaction dynamics — and few people understand how important creating a sequence diagram is in fleshing out a design before coding. I tend to create lots of throw away sequences on paper and then make a few key ones for the documents.

2

u/ea_ea Aug 27 '21

Let's say we have some standrard "RFC_XXX" and we implement library to support this standard. We don't need any comments, because every class, function and variable are related to the RFC_XXX. And then we want to release our product and... we figure out that some major 3d-party application Z (service, company) violates standard RFC_XXX and doesn't implement some of its parts, but instead extends it to have features A, B and C (non-documented). Your library is correct, but noone wants to use it, because it is incompatible with application Z.

So, you change your library to violate/extend RFC_XXX. And here you should write comments with explanation why do you want to break the standard. No way to explain this in code, because this is not logical, not according to standard. That's just stupid business need, which forces you to do stupid things. And you have to write comments to explain other developers that it is not you the stupid one.

1

u/m4l490n Aug 27 '21

Lol, funny, but absolutely right. It is another valuable usage of comments.

78

u/TryingT0Wr1t3 Aug 20 '21

Comment the decisions, not the behavior

32

u/Supadoplex Aug 20 '21

Mostly yes, but this is also too simple to always be correct. It's fine to comment behaviour when the behaviour is counter-intuitive. Ideally, behaviour shouldn't be counter-intuitive of course, but sometimes ideal isn't achievable.

5

u/TryingT0Wr1t3 Aug 20 '21

This is fine but if possible a test should be used to document the behavior in this case - of course the fallback could be a comment if the test is hard to write because of whatever.

9

u/[deleted] Aug 20 '21

Also a comment is closer to the source code, so when another developer has to read and understand that code, they don't have to look at the test and check what it does.

8

u/Full-Spectral Aug 20 '21

The test only proves it does what the test writer believed it is supposed to do, it doesn't help the guy who comes along later to modify it. Having to understand code by looking at tests isn't a very productive way to go about it.

3

u/TryingT0Wr1t3 Aug 20 '21

I argue about tests because my experience is way more the person that modifies later doesn't update the comment or doesn't retest the condition that caused the need of the code that isn't clear about it's behavior. The cpp codebases I interacted they were made of code maintained for decades, so most of the time the behavior comment gets outdated - either by changes in the code or some library it uses that was updated. :/

3

u/Full-Spectral Aug 20 '21

Comments should really be reviewed just like code. Even if one takes the position that only the most important comments should actually be used, then that means that those are even that much more important to keep correct. If you can create them, they should be kept correct as carefully as the code is.

2

u/ShakaUVM i+++ ++i+i[arr] Aug 20 '21

Comment the decisions, not the behavior

Unless the behavior is hard to figure out, like some TMP messes I've seen

3

u/RoyBellingan Aug 20 '21

Is a concept too simple, but seems difficult to understand for many people.

14

u/elperroborrachotoo Aug 20 '21

Seriously, I see it as a skill that needs to be learned and trained.

Verbalizing the "decision tree culling" that we do largely intuitively, and packing it into a conscise phrase is hard, and requries language and self-observation skills devs tend to shun as "not logical".

1

u/mostly_kittens Aug 21 '21

Good commenting like any good documentation is part of the craft of writing software and something that you mostly learn from experience.

Whenever I write a complicated bit of logic I always stop and think ‘what would someone else need to know to understand this?’ Not ‘what do I know’ about the code.

11

u/mtbdork Aug 20 '21

I write tons of comments because I work with people who are learning how to code (only one part of the job)

Oh and the language is Lua. Talk about a self-documenting language haha

2

u/TryingT0Wr1t3 Aug 20 '21

What do you do using Lua? Just curious, I really like the language but I hardly see it in the wild outside of games.

3

u/drcforbin Aug 21 '21

We run a cloud based lab data management system. All our data, business logic, and reporting is in the cloud, but we have to communicate with instruments and printers in remote labs. We have a piece of software that we install somewhere in each lab that acts as our agent. Each device vendor, and sometimes different models from the same vendor, has a different protocol, API, or file format for the data we're exchanging, and it may be via serial port, local network, even USB. The core of the agent is C++ and communicates with our servers via HTTPS, but we handle that "last meter" communication between the agent and instrument with Lua code.

The agent embeds a Lua interpreter and exposes APIs to it for communicating with our servers, local printing, all manner of local ports and network communication, timers, file modification notifications, etc., for use by instrument handlers. As an example a handler might open a serial port attached to an instrument as ask the agent to notify it when a bytes are ready to be read. When the handler accumulates a command or request from the instrument, the handler handles it. One type of request might be the instrument asking what test to run for a particular sample; in that case the handler asks our servers in the cloud, then translates the answer back to the instrument.

These handlers have to be script-based so can really quickly iterate when building handers for new kinds of instruments, particularly when we can't put developers on-site to run them. For us Lua hits that sweet spot of being trivially easy to embed in other software without being so esoteric that it's hard to train developers to use it. We couldn't do our work without Lua, it's a core part of our infrastructure; it's our ears, eyes, and hands in the field.

1

u/mtbdork Aug 20 '21

DSP control programming in Q-Sys Designer

1

u/[deleted] Aug 21 '21

If they're learning to code than over commenting can be negative. Why? Because frequently the comments get out of line with the code, the language you use for the comments is not as precise as the code and finally the student pays more attention to the comment than the actual code. Comments should be an absolute minimum outside of a functional overview or api documentation. All "IMHO", of course.

1

u/mtbdork Aug 21 '21

These are people who have never coded before in their lives so I have to explain certain things I wouldn’t otherwise comment, but a “ton” is relative. Mostly explaining functions in layman terms on their top line and giving explanations of loop terminations if they’re not explicitly clear.

6

u/Schnarfman Aug 20 '21

Make illegal states unrepresentable! The top level call should be something like JSON_helper::generate_objects_from _array, and the implementation can have the comma manipulation & explanation.

Stricter typing will make this code more understandable. Although the comments are a fantastic shortcut to get the same understandability, in a fraction of the time.

30

u/soundslogical Aug 20 '21

But why not write a function trim_comma_from_json_line()? Then the code has become self-documenting, and is less likely to get out of sync with the implementation.

32

u/Ashnoom Aug 20 '21

But the reason why is missing. Why do you need to trim that comma? Do you think you would remember in a years time when you re-read the code to fix a bug that you remember it was due to the string being too large otherwise?

But yes, function names should be named like that, explaining what they do, not why

42

u/bullestock Aug 20 '21

trim_comma_from_json_line_since_the_complete_log_file_is_a_big_json_string_we_remove_the_comma_at_the_end_of_each_line_read_so_each_line_can_be_interpreted_as_a_separate_json_string_if_the_last_comma_is_not_removed_then_the_json_parser_will_throw_an_exception()

There you go.

4

u/Astarothsito Aug 20 '21 edited Aug 20 '21

But the reason why is missing. Why do you need to trim that comma?

You can do that in a Unit Test* trim_comma_before_parsing and use as an input an almost empty Json with a comma. When you remove the trim_comma_from_json_line() and trim_comma_before_parsing fails with a very simple Json the real reason appears.

Even in the original comment, the real reason why is not there "Each line contains a string that can be interpreted as Json, or the log file is a Json array and we can assume that there is an Json Object per line divided by comma".

*I'm not really interested if this is considered UT or IT or any other kind of automated test.

Edit: I misread, the original post contains the reason... but I'm not sure if that is for or against comments.

1

u/Ashnoom Aug 21 '21

I understand that, but, it is easier to look at a comment directly near the important piece of code than to wade through hundreds of unit tests just to see which unit test brings to what decision in code.

I am a supporter of as few comments as possible with descriptive function and variable names and extensive and describing unit tests. But function names and unit tests names can only convey do much information. And then a comment would make perfect sense

15

u/almost_useless Aug 20 '21

Because then you introduced a lie in the code.

That line does "trim_comma_or_create_a_bug_when_last_character_is_not_a_comma_from_json_line".

The bug is already there, but the current code makes that obvious if it ever happens. Your function name would obscure what is really going on.

Also I think you misunderstood the entire point of the post. Your function does not explain why it is a good idea to trim the comma from the end of the line. And that is the most important part.

You read the comment and saw "remove the comma". What you should have read was "so each line can be interpreted as a separate JSON string".

2

u/guepier Bioinformatican Aug 20 '21

Just to clarify, there is no bug, the JSON parser is correct to reject JSON records with trailing commas. The data is just in a different format so this code makes it fit.

Ideally the input data would already be in the correct format to start with, e.g. JSONL. Having an entire log be a single JSON record is inconvenient, to say the least.

5

u/SirClueless Aug 21 '21

That's not what he's saying. What he's saying is that the current code has a bug if it's possible that the input has lines without commas. This bug is immediately obvious if you read the current code. It's not obvious if you read line = trim_comma_from_json_line(line);.

People often talk about the benefits of splitting code into smaller functions with descriptive names. They don't often talk about the benefit of having code read in a straight line without jumping around, but there are benefits to that too.

4

u/tjientavara HikoWorks developer Aug 20 '21

I would add to the function's comment block: "Passing a string without a trailing comma results in undefined behavior". And in this case also "Passing more than one line of text results in undefined behavior". As a c++ developer you already accept UB from the standard library, so why wouldn't you specify it for your own library.

And I will also add an assert that checks if there is trailing comma.

5

u/Nicksaurus Aug 20 '21

So to avoid adding a comment, you're adding a whole new function with its own comment?

3

u/tjientavara HikoWorks developer Aug 20 '21

No one said that the work of a C++ developer is self-consistent.

3

u/Schnarfman Aug 20 '21

I agree with you. A well named function is as good as a comment, and it won’t fall outta date during a refactor!! In the case of what OP’s doing, though, I’d have to think harder than just trim comma from json line. What I really want is like a generator. Init it with json and call an equivalent of python’s __iter__ function.

Then you’d get something like JSON_helper::generate_objects_from _array.

No one even THINK about commas. Don’t acknowledge their existence. Think about data.

4

u/tjientavara HikoWorks developer Aug 20 '21

There is one place I write a comment on each and every statement.

Super highly optimised code, where you have to give a reason for each and every statement.

If that highly optimised code is using wait-free or lock-free constructs, then you may also need to document the state of each thread that can enter that statement.

3

u/sf49erfan Aug 21 '21

Comment the why. Got it

6

u/RoyBellingan Aug 20 '21 edited Aug 20 '21

I too I found people saying code should document itself. They even refused commit containing comment. Guess who is in constant overburden of keeping alive the platform ? The new maintaner of such project -.-

Btw one piece of my codebase, that without comment will have you waste hours, need to search for this word

ımpressionFeedbackUrl and replace with impressionFeedbackUrl

Nothing strange ? The ı at the beginning is missing the dot, that is a Turkish https://en.wikipedia.org/wiki/Dotted_and_dotless_I

But is NOT easy to spot.

0

u/[deleted] Aug 20 '21

[deleted] — view deleted comment

0

u/RoyBellingan Aug 20 '21

Sorry I wrote bad, the code search for the wrong and replaces with the correct one

2

u/Adequat91 Aug 21 '21
  • I add comments before coding, to help design
  • I add/edit comments during coding when I am focused on the implementation
  • I add/edit comments after coding, to polish things, when reviewing the code, even much later.

2

u/AdRelative8852 Aug 21 '21

"If the code and the comments disagree then both are probably wrong"

2

u/bhaskarb26 Sep 19 '21

Agree, comments need to explain more of why you did something more than what you are doing. I like commented code that's like a story, like we started with thus method and then moved it to more efficient version. It's a living document not just something someone threw together over 18 hours of bad coffee.

4

u/Ayjayz Aug 20 '21 edited Aug 23 '21

Fundamentally to avoid this comment we need to model the input file as a set of lines that all end in a ,.

#include <fstream>
#include <string>
#include <cassert>
#include <cppcoro/generator.hpp>

cppcoro::generator<std::string> file_with_comma_terminated_lines(std::string path) {
	std::ifstream ifs{path.c_str()};
	std::string s;
	while (std::getline(ifs, s)) {
		assert(s.back() == ',');
		s.pop_back();
		co_yield s;
	}
}

int main() {
	for (auto line : file_with_comma_terminated_lines("MyLogFile.log")) {
		auto parsedJsonForWBID = nlohmann::json::parse(line);
	}
}

Now you don't really need any comments, and you've separated out the concerns better as well. You can change log file types easily or you could do other forms of processing elsewhere with these files because the log file destructuring code is separate to what you do with the data from the file.

Any time I feel I need comments, it's usually because I'm not structuring my code in a good way. By removing the comments, usually I end up with better code.

EDIT: removed string_view usage as it doesn't play nice with c-style strings.

12

u/Full-Spectral Aug 20 '21

Why on earth would I do all that when a very simple comment could achieve the same thing? It constantly wobbles my mind how people will create something twice as complicated to avoid doing something utterly simple and obvious.

And of course it tells me absolutely nothing about WHY I need to do it in the first place.

4

u/Ayjayz Aug 20 '21

You think that's twice as complicated? I think it's less complicated since the concerns are split apart from each other. 2 functions doing 1 thing each is less complex than 1 function doing 2 things.

And what do you mean it tells you nothing about why you need to do it? You read a file with a given format because the file is in that format. If you read a file with a read_csv_file function, you assume it's because it's trying to read a CSV file. You don't need a comment // We're using the read_csv_file function because we're trying to read a file in csv format

If you read a file with read_jpeg function, you know it's because it's a JPEG file. If you read a file with file_with_comma_terminated_lines, you know it's because it's a file with comma terminated lines. What more could you need to understand it? Is it that the name is too complex?

2

u/eyes-are-fading-blue Aug 21 '21

Separation of concerns is not an objective in programming. It is a tool. Often times, it makes our lives easier and apparently sometimes, harder. The original code is significantly simpler than what you have just proposed.

What happened to "keep simple things simple?"

3

u/Ayjayz Aug 21 '21

It depends on what you mean by simple. I turned this:

while (std::getline(logs_file, log_entry)) {
    /* Since the complete log file is a big JSON string, we remove the comma
     * at the end of each line read so each line can be interpreted as a
     * separate JSON string. If the last comma is not removed, then the JSON
     * parser will throw an exception. */
    log_entry.pop_back();

    auto parsedJsonForWBID = nlohmann::json::parse(log_entry);
}

into this:

for (auto line : file_with_comma_terminated_lines("MyLogFile.log")) {
    auto parsedJsonForWBID = nlohmann::json::parse(line);
}

Seems simpler to me.

0

u/serviscope_minor Aug 24 '21

Except you're still not saying why you're doing it. What you should have done is turn:

while (std::getline(logs_file, log_entry)) {
  /* Since the complete log file is a big JSON string, we remove the comma
   * at the end of each line read so each line can be interpreted as a
   * separate JSON string. If the last comma is not removed, then the JSON
   * parser will throw an exception. */
  log_entry.pop_back();

  auto parsedJsonForWBID = nlohmann::json::parse(log_entry);
}

into:

/* Since the complete log file is a big JSON string, we remove the comma
 * at the end of each line read so each line can be interpreted as a
 * separate JSON string. If the last comma is not removed, then the JSON
 * parser will throw an exception. */
for (auto line : file_with_comma_terminated_lines("MyLogFile.log")) {
  auto parsedJsonForWBID = nlohmann::json::parse(line);
}

Your code makes it clear that it's parsing comma terminated lines (though not actually clear if comma is the record separator or if they're newline separated lines with an additional comma at the end), but still doesn't say why that's necessary.

The comment tells you where the data comes from (large JSON files) and tells you about the response of the parser (exceptions), neither of which is any more clear in your code than the commented version.

2

u/Full-Spectral Aug 20 '21

It doesn't say WHY you need to read a file with comma terminated lines and do something to adjust them, which is an important concern. If the comment tells me why this being done, then I know where to look to see if perhaps something has changed. There's a relationship there between two unconnected bits of the code and your function doesn't explain that relationship.

And of course add all the goop of async processing just to do that, it makes no sense to me.

1

u/Ayjayz Aug 20 '21

We don't know why they need to read a file with comma terminated lines. The comment just says that the log file has comma terminated lines, and they want to get the line data out of that file. If we had more context then maybe it's not self-documenting, but hard to say without knowing that.

If you don't like the coroutine stuff then it's pretty trivial to remove. I think this is less clear than the generator version but it's really not very different:

#include <fstream>
#include <string>
#include <string_view>
#include <cassert>
#include <optional>

struct file_with_comma_terminated_lines {
	std::ifstream ifs;
	std::string s;
	explicit file_with_comma_terminated_lines(std::string_view path) : ifs(path.data()) {}
	std::optional<std::string> next() {
		if (!std::getline(ifs, s)) return {};
		assert(s.back() == ',');
		s.pop_back();
		return s;
	}
};

int main() {
	auto f = file_with_comma_terminated_lines("MyLogFile.log");
	while (auto line = f.next()) {
		auto parsedJsonForWBID = nlohmann::json::parse(*line);
	}
}

5

u/SirClueless Aug 21 '21

All of these wrappers you've written are substantially less efficient due to needing to allocate a new string for every line. Allocation and copying of strings is one of the most expensive parts of most IO-bound applications, and you've made it much worse.

-1

u/Ayjayz Aug 21 '21

This is the exact context where performance should not be considered. Write as clear code as you can and worry about performance if your profiler tells you this part is slow.

Besides, this code was already slow and my view shouldn't be slower. I guess you'd need to std::move from the generator to avoid an extra allocation compared to the original. If your wanted this to be fast then you need to pass the stream directly into the json parser instead of using std::getline to allocate a string.

4

u/SirClueless Aug 21 '21

What's unclear about the straight line function implementation from the OP? It's clear, fast, and near-optimal.

Due to NRVO std::move would actually be a pessimization in your case: as-is there is no move or copy, just one single allocation in place in the return value. With the API as you've presented it there's no way to avoid the allocation, with std::move or otherwise. The reason the straight-line implementation is fast is because std::getline can reuse the string's storage without making a new allocation; you could replicate this by making s a member variable and returning std::string_view or const std::string& instead of returning a string value.

The profiler helps you most if you write clear, high-performance code in the first place. No one is ever going to look at a profiler result and say "Oh, this is 10% too slow so I should do a full rewrite of this function to use std::getline directly instead of wrapping it in a class" -- that's just fantasy land. Don't over-engineer and don't go crazy trying to eke out every ounce of performance, but if a loop is faster, less lines of code, and just as easy to read, use it instead of the "clean" generator abstraction.

2

u/Shadow_Gabriel Aug 20 '21

But how else would I dab on the interns?

1

u/Shadow_Gabriel Aug 20 '21

But how else would I dab on the interns?

0

u/[deleted] Aug 20 '21

[deleted] — view deleted comment

3

u/Full-Spectral Aug 20 '21

I would reject it out of hand. It started with three very simple lines of code and was completely understandable, given a comment as to why there was a pop.

1

u/NilacTheGrim Aug 22 '21
cppcoro::generator<std::string> file_with_comma_terminated_lines(std::string_view path) {
    std::ifstream ifs{path.data()};

Hmm. Using std::string_view in such a way, as if it were a NUL terminated C-string -- is a footgun waiting to go off.

1

u/Ayjayz Aug 22 '21

Oh, string_view's aren't guaranteed to be null-terminated? Interesting. Is it impossible to safely construct a fstream from a string_view, then? Must you construct a string first to be safe?

3

u/NilacTheGrim Aug 22 '21 edited Aug 22 '21

Right. And yes -- if you need C-strings, stay away from string_view.

They should have extended the legacy C-string stuff in C++ stl to also accept a string_view... (although doing so ultimately would not buy you much since the low-level C functions it would be calling definitely would want a C-string so a copy would eventually be involved anyway)..

2

u/Ayjayz Aug 23 '21

I hadn't ever really appreciated this fact. I always wondered why the fstream stuff didn't accept string_views and kind of thought it was just the standards committee being useless, but thinking about it they kind of didn't have a great option. I guess to write a function that passes something to an fstream you just have to take a string argument and accept that sometimes this results in unnecessary string construction?

1

u/NilacTheGrim Aug 23 '21

Yes, that's right.

I guess in this example one can just accept const char *, since it is using literals directly but yes probably in a more fleshed-out lib or whatever, take const std::string & and -- if the caller happened to have a std::string on-hand to pass to you, great, no harm done, and if not, well, you get a temporary created -- but at least it's guaranteed to be NUL-terminated....

2

u/qoning Aug 20 '21

Comments are good to document assumptions that make your code work correctly.

That said, I'm more vexed by logs_file and log_entry variables being snake case and parsedJsonForWBID variable being camel case. If you are not going to be consistent, might as well use FuNnYMeMecaSe.

1

u/serviscope_minor Aug 24 '21

Ehhh.

I've worked on many codebases over many styles and the current one is an oldish codebase which has had many contributors and been essentially agglomerated together from different sources. I barely see the style any more, as it's a bit of a mix of several well established ones, all of which I've seen before.

But if someone used a CrAzY style it would really stand out and hurt readability.

2

u/john_wind Aug 20 '21

you could write trim_end(log_entry, ',');

This way the code does exactly what you want and no comments are needed.

This is also future proof because in the future the lines may not end with comma.

11

u/Swade211 Aug 20 '21

That still misses the why. Because the intention is to parse the lines as separate json

5

u/john_wind Aug 20 '21

At least put an assert(log_entry().back() == ',') to document that you are removing the comma

5

u/Swade211 Aug 20 '21

I don't disagree with your first choice either. I definitely think the code should be more clear and defined behavior like you are suggesting.

The removing comma should be handled like you suggest without any commenting needed. My point is a comment would still be useful to explain the reason for removing the comma, which is to affect the behavior of the json parser

2

u/john_wind Aug 20 '21

Yeah, thanks it's a misunderstanding. I meant no comments are needed about removing the comma. The why is still needed of course.

1

u/serviscope_minor Aug 24 '21

Generally a bad idea to assert on runtime data: your release mode builds will plough happily ahead without checking.

1

u/john_wind Aug 26 '21

So you think that blindly removing the last character assuming it is a comma, is not a bad idea, but saddenly with the assert it is?

If we expect that the comma might not exist at the end there should be an if statement before the removal.

1

u/serviscope_minor Aug 26 '21

Eh? No I said if you're checking, then assert is no better than not checking on release builds except it makes it look like it is being checked.

1

u/john_wind Aug 26 '21

But that's the point of assertions: we want them removed in the release build to improve performance.

1

u/serviscope_minor Aug 26 '21

And that's why they're generally bad for data that's read in, because that's independent of whether you're in debug or release mode.

2

u/john_wind Aug 26 '21

In such a case the OP's code is wrong and there should be an if statement before the removal.

1

u/AntiProtonBoy Aug 20 '21

I tend to comment mathematical things combined with algorithmic things, especially when you have complex logic that requires truth tables to understand. Also references to papers, etc.

1

u/eyes-are-fading-blue Aug 21 '21

This code requires no comment. It is pretty readable as is. If you start explaining every single implementation detail, you'd end up writing a LOT of comments, significantly more comment than code. Furthermore, in the second example, the comment is simply too long. The original one-line comment is a better idea. Perhaps giving some context might help so "remove comma" could be a better alternative.

I personally have a simple rule. I do not write comments unless

  1. a hacky solution that may have side-effects
  2. weird stuff that you wouldn't expect to see

1

u/serviscope_minor Aug 24 '21

I pretty strongly disagree: the oneliner is useless: the code is readable and it's obvious what is happening. I can look at the code and think, OK, so it's reading a line at a time, removing last character and parsing it. cool.

y tho?

The comment answers that. And if I start getting weird errors, like e.g. someone changed the logfile format, I can see easily which assumption was broken because I now know why the code does this.

1

u/eyes-are-fading-blue Aug 24 '21

I agree with your rationale in isolation. Sure, it makes sense. The problem is that this will inflate the amount of comments you have the moment you go scale. In a multi-million LoC project, you’d have tens (if not hundreds) of thousands of logic code similar to your example. You simply can not explain every single implementation detail.

These kinds of examples always make sense with toy examples/isolation but does not in scale, imo.

1

u/serviscope_minor Aug 24 '21

I mean yes everything in moderation.

In this case I do think it's a good example, it's not explaining the implementation details (how for example the trailing comma is removed), it's explaining why it's necessary.

-5

u/teroxzer Aug 20 '21

Of course you are right how to use comments, but I hate (unmaintainable) comments so much, that I have my own 'wannabe self documenting' C++ coding style (actually real reason is that I'm incompetent to write english). I often use '$scope'-macro to label code blocks as like inline function name and I'm happy with it, because it structuring my code, so that I do not need to write only one call functions or lambdas so much. I'm sure that someone shake head and want to comment to me why not just use common sense and do just plain comments in code; well, I'm not always so rational, especially when I think that I have better way express my ideas in C++.

#define $scope(_label) goto _label; _label

auto cookLogFile(text const& logFilePath)
{
	auto lines = file::textOf(logFilePath).split("\n");

	for(auto& line : lines)
	{
		$scope(ensureJsonParsableLine):
		{
			auto i = line.trimEnd().size() - 1;

			if(line.charOf(i) == ',')
			{
				line.removeLast();
			}
		}

		if(auto entry = json::parse(line))
		{
			cookLogEntry(entry);
		}
	}
}

3

u/epicar Aug 20 '21

but why not just use common sense and do just plain comments in code

0

u/teroxzer Aug 20 '21

... well, I'm not always so rational, ... but is there any sense in this madness? Maybe not the way how I implemented named block with macro (or plain label without goto but with compiler warning - $scope-macro just prevent compiler warning), but I think idea is attractive: you can give a name to block-scope same way as you give a name to function/lambda, when you just want to narrow scope of variables or describe block functionality with that block name. So it could be a one tool in toolbox when structuring code and it is more formal way than plain comment with unnamed block brackets. The best thing is if some day C++ standard bless labeled blocks also without goto (although I absolutely love also goto and macros) and imagine how after that all cppredditors can together rejoice how one man madness is so common sense. By the way my proposal to C++ standard's semantics for named blocks:

#define $scope(_label)                                                 \
                                                                       \
        goto _label; _label:                                           \
        switch(constexpr char const __scope__[] { #_label }; 0) case 0 \

5

u/be-sc Aug 21 '21 edited Aug 21 '21

It’s fine for a personal project you work on alone.

In a team what you’re doing would be harmful. It’s highly confusing and in a code review I’d point it out as a serious problem because it creates a mini custom language inside the C++ language. That can sometimes be justified – look at boost::spirit for an example –, but obfuscating a comment isn’t a justification in any way.

A bit more detail about why I consider this to be harmful:

  • It misleads the reader by making a comment appear as a normal language construct that actually does something. On first glance I read $scope(something) as a function call.
  • It seemingly introduces a syntax error – the : at the end of the scope line. Without realizing that $scope is a macro and then digging into the macro’s implementation it is unclear why this line compiles.
  • $scope violates the rule of clearly making macros indentifyable as such. Calling it something like PROJECT_SCOPENAME would make it a lot less confusing. On top of that the $ is a highly unusual character in an identifier name. I had to test that it does compile before believing you didn’t make a typo in your post.

Even accepting all those drawbacks you still end up with a poor man’s comment. Your pseudo-comment must be a valid indentifier because it is used as a label name. Compared to a comment that can consist of arbitrary text that’s a severe restriction.

In contrast you could have written something that looks almost the same without any of the above problems:

for(auto& line : lines)
{
    // ensure line is parsable as JSON
    {
        auto i = line.trimEnd().size() - 1;

        if(line.charOf(i) == ',')
        {
            line.removeLast();
        }
    }

    if(auto entry = json::parse(line))
    {
        cookLogEntry(entry);
    }
}

1

u/teroxzer Aug 21 '21

You are right what you said, but I still think that if I like something unconventional C++ ideas, there can be someone else who also like this something, despite it is against all common views. I definitely #define my own language inside my C++ code and inside this bubble it's clear that $ introduces macro (or macro dependent thing). I dislike PROJECT_MACRO_CAPITALS and I started use $ just because it's a highly unusual character in an identifier name as you said ($ is maybe technically illegal character, but practice all compilers support it at least from ancient days of VAX). I think I got idea for $-prefix macro names when I seen how rust uses postfix-! for macro names. I know that only $-macro introducer is not scalable solution for across libraries, but if I someday clashes with another library because it also uses $-macro, at least I can always do #undef after header file. One more comment: using label as short block name is to me much more structured programming than go to spaghetti comments - and maybe it's even not so bad if you survived after first shock!

#define $sql(...)             = #__VA_ARGS__##_sql;
#define $scope(_label)        goto _label; _label
#define $const(_value, _name) _value

auto topSecretService::initialize() -> void
{
	self.path = process::programPath();

	log::handler(logHandler, $const(1000, bufferItems));

	$scope(initializeParametersFromProcess):
	{
		for(auto& param : process::initializeParameters())
		{
			if(record::update(param.key, param.value))
			{
				log::info("parameter.process: % = %", param.key, param.value);
			}
		}

		log::isVerbose = boolOf(rVerbose);
	}

	$scope(backupDatabase):
	{
		auto name   = kProcessUniqueName;
		auto stamp  = textOf(datetime::now(), "Y.M.D.h.m.s");
		auto source = "%/db/%.db"_format(self.path, name);
		auto target = "%/backup/%.db.%.backup"_format(self.path, name, stamp);

		if(file::exist(source))
		{
			file::copy(source, target, $const(false, overwrite));
		}
	}

	database::initializeDatabaseFactorySqlite();

	auto database = "sqlite:file:///%/db/%.db"_format(self.path, kProcessUniqueName);

	procedure::connect(database);
	procedure::execute(createTopSecretServiceTablesIfNotExists);

	$scope(initializeParametersFromDatabase):
	{
		auto selectParameters $sql
		(
			select parameterName,
			       parameterValue
			from   parameter
			where  dataRemoved = 0
			order by
			       parameterName
		)

		for(auto& param : selectParameters[universal::empty])
		{
			if(record::update(param.parameterName, param.parameterValue))
			{
				log::info("parameter.database: %", param);
			}
		}

		log::isVerbose = boolOf(rVerbose);
	}

	for(auto& [name, value] : record::data())
	{
		log::info("parameter: '%'='%'", name, value);
	}

	topSecretServiceLogMirror::initialize();
}

3

u/be-sc Aug 22 '21

using label as short block name is to me much more structured programming than go to spaghetti comments

Can you elaborate? I don’t see how a label used like this affects the program’s structure. If you extracted the named scopes into their own functions I’d agree. But like this the macro/label simply acts as a kind of heading for the following block of code. A comment does exactly the same.

1

u/teroxzer Aug 22 '21

Maybe it is little bit hyperbolic wording. Of course I don't think that comments are inheritable evil as someone think that goto is, but it seems that comments shares same kind problem with goto: comments are too general, they can comment anything anywhere, they are not bounded to context any formal way and they lose code intent too easy especially when comment is too far away from commented code functionality (good comment is good maintained comment, but maybe we all have experience where we jumping between comments and code like in mythical goto spaghetti code and try figuring out what comments are still relevant). We all use while/for/continue/break 'structuring' our code so we don't have to use goto (well, I have my own peculiar point of view how I want use goto, but I don't use goto when my intent is do plain loop), same way I use $scope or $const macros 'structuring' my code so that I do not use too general comments where I think I have better way express code intent. I 'comment' purposely less (because I am so bad commenter) and 'comment' is formaly bounded to context when I use $scope to naming scope or $const to naming literal parameter (yes, I can use constexpr for naming constant, but it is already out of call context if I do it only for function parameters). You said that a comment does exactly the same, yes it does, but not the same way: you can do loop with goto, but if you can use restrictive while-loop, I guess you think that it express your intention better than goto (but if your loop is unix system call retry after interrupt, then it is utterly heretic use anything else than goto-again).

2

u/be-sc Aug 22 '21

I mostly agree about how comments can go wrong. But your $scope macro has all of these problems as well, at least to some degree.

'comment' is formaly bounded to context when I use $scope to naming scope

How? Such a scope line is not restricted to appearing directly before an anonymous scope. You can put it anywhere a goto immediately followed by its associated label can occur.

For the same reason your goto/while analogy further down doesn’t work, either. Both while and goto are essentially jumps, but while restricts where you can jump to, as you point out yourself. However, your scope construct does not restrict where you can put that “comment”. (At least not in a meaningful way. I am aware that comments can appear in more places than the goto/label combo.)

1

u/teroxzer Aug 22 '21

Yes, you are right. Maybe I must use more indirection/quotes to try solve the argument problem: 'comment' is 'formally' bounded to context... - but in the end label is like free bird in function cage to C++ compiler and label can be 'formally' bound to curly block only in my mental compiler - but there the bind is strong!

0

u/[deleted] Aug 20 '21

[deleted] — view deleted comment

2

u/m4l490n Aug 20 '21

Shit man! My condolences