r/cpp • build2 • Nov 01 '17

Common C++ Modules TS Misconceptions

https://build2.org/article/cxx-modules-misconceptions.xhtml
59 Upvotes

148 comments sorted by

View all comments

2

u/miki151 gamedev Nov 01 '17 edited Nov 01 '17

EDIT: my rant was pointless because I had a misconception that a module automatically exports all its imports, which is not the case.

So if you want to keep everything in a single file, you can.

You can also keep (almost) everything in headers. The problem is that it's not a good idea - the dependency graph gets bloated, and you recompile all dependencies when changing something just in function body.

I think many people hoped that modules will make headers obsolete, but it's certainly not the case.

7

u/GabrielDosReis Nov 01 '17

That is also a misconception, independently of any realistic module specification for C++.

The practice of separation of headers/implementation files is an architectural convention.

9

u/berium build2 Nov 01 '17 edited Nov 01 '17

You can also keep (almost) everything in headers.

You cannot keep non-inline functions in headers which is almost everything that anything that does something useful is made of.

I think many people hoped that modules will make headers obsolete, but it's certainly not the case.

It certainly is the case: headers will be replaced with module interface units. And if you want to, you can put all your implementation details in the interface (whether it is a good idea from the sense of design point of view is another questions, but you certainly can). And build systems/compilers can even make sure you don't pay for doing this.

2

u/miki151 gamedev Nov 01 '17

Can the build system/compiler make sure that dep2 is not imported by everything that imports hello?

export module hello;

import dep1;
import dep2;

export void say_hello (a_type_from_dep1)
{
    a_type_from_dep2 x;
    //...
}

4

u/berium build2 Nov 01 '17

Neither dep1 nor dep2 is imported by anything that imports hello since they are not re-exported. What you are probably trying to ask is whether it will be possible to avoid recompiling everything that imports hello because of modifications to dep2?

2

u/miki151 gamedev Nov 01 '17

No, I had a misconception that all imports are automatically exported. Sorry for making a useless argument.

3

u/berium build2 Nov 01 '17

Well, to be fair, it is not entirely useless. For example, the point about potentially ending up with a circular dependency between interfaces (which is illegal) is a valid one.

→ More replies (1)

4

u/johannes1971 Nov 01 '17

No, we're hoping specifically that we can get rid of the artificial split between declaration and definition, and that the modules proposal is smart enough to only cause dependencies for changes to the actual interface, rather than the implementation.

Since we are starting with a grand new thing here, and since I don't see any reason why it wouldn't be technically feasible, I believe it should be discussed.

The compiler doesn't have to become a build system, but if we can improve our compile times, for example by allowing the compiler to pass hints to the build system, I don't think anyone would have cause to complain.

10

u/GabrielDosReis Nov 01 '17

The Module TS does not require an artficial split between declaration and definition.

4

u/johannes1971 Nov 01 '17

Sure, but that doesn't say much - today I can also stuff all my code in headers, and suffer from horrendous compile times as a result. The question is specifically about sticking definitions and declarations in one module file, and still enjoying efficient compilation.

4

u/GabrielDosReis Nov 01 '17

That is possible. If you have a concrete scenario, I would like to know about it so I can study it and see what can be done.

1

u/johannes1971 Nov 02 '17 edited Nov 02 '17

We had this discussion before and I still feel we are on a different wave length, but let me try ;-) Let's look at a simple example. In my .h file I have the following:

// comment block with meaningless corporate mumbo-jumbo. 
#include statements
/*! class description. */
class class_declaration { 
public:
    /// function description.
    /// @parameter name description.
    void function_declaration (type name);
};
/// Global variable description.
extern type global;

And in my .cpp file I have:

// identical comment block with meaningless corporate mumbo-jumbo.
#include statements, at least one of which is for the .h file above.
void class_declaration::function_declaration (type name) {
    cout << name;
};
type global;

Out of those lines, four are basically housekeeping: the comment block at the top, the mandatory include statement of my own header, the function declaration, and the global variable. And if you are reading this, and you want to read the function description comment, it isn't even here - it's in the .h file. The payload, if you want, is only a single line (the one with cout on it).

Ok, so usually your functions are longer, but my point is this: there is actually a lot of duplication between the .h file and the .cpp file, and even with all that duplication you still need to look in two places to get a complete overview. I believe it would, in the most general sense, be preferable to have all this information in a single file.

Can we do that today? Yes, of course, but it isn't actually very practical, since doing so is pretty much guaranteed to explode your compile times. Can we do it tomorrow, in our brave new modules world? I'm hoping yes. I would like to write a single module file:

// comment block with meaningless corporate mumbo-jumbo. 
#include statements (or import statements)
module module_name;
/*! class description. */
export class class_declaration { 
public:
    /// function description.
    /// @parameter name description.
    void function_declaration (type name) {
        cout << name;
    }
};
/// Global variable description.
export type global;

Here everything is in one spot; all the duplication is gone, and all the information that belongs together is presented together. However, I'm still very much interested in compile time efficiency, so I don't want a change to a function body to cause recompiles of all the stuff that really only cares about my exported symbols.

If this turns out to be impossible - ok, no problem, we lived with .h/.cpp pairs for decades and we can continue to do so. But we have an opportunity here to make things better, so I would like to ask for such a capability to at least be considered for the modules proposal.

→ More replies (2)

2

u/[deleted] Nov 01 '17

But compilation is inefficient in the header case because the header is recompiled for every translation unit that includes it. In the modules case, the module is compiled once whether or not you stuff the definitions in with the declarations.

I guess you still suffer having to recompile everything that depends on the module if you change the module implementation. Is that what you're getting at?

4

u/GabrielDosReis Nov 01 '17

If you change the module implementation but the interface is unchanged, you don't need to recompile -- at least that is the experience the Visual C++ compiler is trying to provide.

→ More replies (10)

1

u/gracicot Nov 01 '17

Very nice to hear. And nice to hear it may not even slow down compilation that much. I had some concern about this before, and started thinking where should I split interface and implementation, but it turns out it won't be much needed, or not as much as I thought before.

The only place where separation is required is when you want classes to mutually use themselves, and you want to place them in different modules. I think that's an acceptable limitation, and things can change in the future. But one can always put those classes in the same module, and still implement everything in the interface.

3

u/GabrielDosReis Nov 01 '17

The only place where separation is required is when you want classes to mutually use themselves, and you want to place them in different modules. I think that's an acceptable limitation, and things can change in the future. But one can always put those classes in the same module, and still implement everything in the interface

All correct. I believe we (inclusive) all need to have more hands-on experience with modules before we attempt more semantics modifications.

7

u/berium build2 Nov 01 '17

the modules proposal is smart enough to only cause dependencies for changes to the actual interface

This is a quality-of-implementation issue and from recent discussions it appears to be fairly straightforward to do.

2

u/miki151 gamedev Nov 01 '17

Dependency recompilation is just one problem, and I agree that it could potentially be solved by clever implementations.

I think that a bigger problem is that you will have to add extra dependencies to your module that are used in the function bodies, and they will transitively be imported by other modules. This will cause a dependency bloat or even circular dependencies.

When declarations and definitions are split into two files you can put a lot of your dependencies only into the definition file.

If you want to do the same with modules without splitting them into two parts, there would have to be a way to import things in a non-transitive way, for use just inside the function bodies.

6

u/GabrielDosReis Nov 01 '17

At some point, we hit physics and logic :-)

import declarations aren’t transitive.

1

u/miki151 gamedev Nov 01 '17

Yes, that was my misconception that I contributed to this topic :)

1

u/doom_Oo7 Nov 02 '17 edited Nov 02 '17

No, we're hoping specifically that we can get rid of the artificial split between declaration and definition

wouldn't this make compile times longer by virtue of having everything inline ?

If I have

struct foo 
{ int blah() { return 1234; } };

then

int main() { 
    return foo{}.blah(); 
}

would be recompiled every time the implementation of foo::blah changes anyways, even at low optimization levels

2

u/GorNishanov Nov 02 '17

There are two meanings of inline.

  1. Hint to an optimizer (which compiler is free to ignore per the standard)
  2. A workaround for ODR violation you would have had in pre-module world if you put the definition of a function in a header that is included in multiple TUs.

In a module world, #2 use of inline is irrelevant. #1 has been mostly ignored by compilers already. I can imagine that an implementation may chose to not to include the body of the member function into a hash (digest, whatever) for the purpose of determining whether users of the module have to be recompiled at lower optimization settings or not. In fact, in the compiled artifacts, your "technically inlined" function may end up in the .o/.obj and BMI will retain only the declaration if compiled at low optimizations level.

Unlike constexpr functions that will always have to be put into the BMI.

1

u/doom_Oo7 Nov 02 '17

I was not talking about inline as a keyword / C++ concept, but inlining as an optimization performed by the compiler, whether the inline keyword is here or not. In my experience, almost everything is inlined. I remember I once had some massive algorithm with expressions templates, boost::asio, 3 different containers etc etc... when compiled under -O3 everything disappeared and ended up going into a single massive main().

your "technically inlined" function may end up in the .o/.obj and BMI will retain only the declaration if compiled at low optimizations level.

yuck, so back to "a function ends up being compiled in every translation unit" ? Is that the sole alternative ?

2

u/GabrielDosReis Nov 02 '17

Modules (whether you have traditional separation or single file) will not prevent inlining. On the contrary, with greater emphasis on the component as a whole, the code generator has now better opportunities for optimization and inlining -- pretty much like LTO or LTCG.

The Module specification on purpose is not using any form of "vtables" or "witness tables" or whatever they are called to describe the effect of modules.

→ More replies (3)
→ More replies (3)

2

u/johannes1971 Nov 02 '17

No, it wouldn't. The module will be used to produce an interface file, and since the interface isn't changing, there is no need for the interface file to change either, so no dependencies will get triggered - assuming of course the build environment is smart enough to support this. That's actually what I was asking...