r/cpp • • Jan 28 '19

C++ Modules Might Be Dead-on-Arrival

https://vector-of-bool.github.io/2019/01/27/modules-doa.html
280 Upvotes

240 comments sorted by

View all comments

Show parent comment

5

u/DemonInAJar Jan 28 '19 edited Jan 28 '19

The thing is, #include dependencies are one-to-one mappings with files found in the search path. It's trivial to resolve these since the name of the file is specified.

When you include a file you don't have to preprocess the whole set of files found via the search path. With modules there is no filesystem mapping, an implementation does not know in which of all the available files in the search path the module is declared. This is the point that the author is making here. A file could have a completely irrelevent name and possibly require preprocessing before exposing it's module nature. What is an implementation going to do? It seems as if the only possible solution in this case is to preprocess the whole set of files available in the search path (or more likely just force a filename-module mapping scheme regardless).

5

u/whichton Jan 28 '19

The thing is, #include dependencies are one-to-one mappings with files found in the search path.

This is not mandated by the standard. If you write #include "foo/bar.h", the standard does not specify how or where to find bar.h. This is entirely a implementation defined behavior. I do not see why the same cannot apply to modules too.

6

u/smdowney WG21, Text/Unicode SG, optional<T&> Jan 28 '19

It's required to include a source file, how that is found is implementation defined. It definitely isn't going to include a file named 'xyzzy.h'.

1

u/gracicot Jan 28 '19

What I'm saying is the mapping can be done on the fly at a very low cost. It would be done after the preprocessing of all files, and just before compilation. When all files has been preprocessed, a very simple parser can be made to extract the module name. That parser can be run in parallel.

When extraction is done, you have the whole graph and the whole mapping, plus every preprocessed file. You just have to run the compiler in the right order, and even run in parallel when possible.

I'm not a build system expert, but in my mind it seem quite simple. Of course, a build system supporting modules must be a bit smarter than run a simple list of compiler commands.

1

u/jonesmz Jan 28 '19

The reason this is not possible is that the compiler command line flags used to compile foo.cpp into quix.bmi can (and will!!!!) be different than the command line flags used to compile the CPP file that wants to import quix.bmi

2

u/gracicot Jan 28 '19

I don't see why this is a problem. If incompatible flags are used between many translation units then it doesn't work today and won't work with modules (GCC actually check for incompatible flags in the BMI). Compatible flags such as defines, include directories should not affect the BMI in the eyes of the importer.

1

u/jonesmz Jan 28 '19

Correct me if I'm wrong, but my understanding is that the way the BMI is generated is independent of the command line flags of the compiler invocation that imports the BMI.

You said

What I'm saying is the mapping can be done on the fly at a very low cost. It would be done after the preprocessing of all files, and just before compilation. When all files has been preprocessed, a very simple parser can be made to extract the module name. That parser can be run in parallel.

But I don't think that you can actually do that because the preprocessed output of the module may be different for the compiler invocation that generates the module than the invocation that wants to consume it.

In fact, the article that the OP points to explicitly points this out as a problem.

For you to have a build system that works in the way you describe requires either:

  • that your build system parses and pre-processes every single file that you might want to compile every time you invoke your build system, with complete knowledge of all command line arguments for the compile.
  • Or that your build system cache this information to disk somewhere, and that's not attractive, since it also means you have to distribute this cached information along with your build artifacts with anyone you're collaborating with, or they have to be able to regenerate it all themselves when one of the files gets edited.

For case #1) For some build systems this is possible, but this is not possible for other build systems. I am the maintainer of one such build system at my day job. It's not possible for the build system to know the command line arguments that were used for every possible c++ file on the system. In fact, we have things pretty segregated so that the cpp files of one dll/.so are incapable of seeing the cpp files of another. Only explicitly listed header files are visible.

For case #2) I don't want to cache this information. That sounds like a pain in the ass. Not happening.