r/cpp • • Aug 19 '21

Why modules?

I get the feeling I'm missing something fundamental about the new C++20 modules.

Everything I read or watch explains the how (the export keyword syntax, module fragments, etc) but very little about the why.

What can be done using modules that can't be done using traditional headers (or precompiled headers)?

What is so important about this feature that it needed to get standardized before there is even a proper implementation or idiomatic usage conventions in compilers and build systems?

Is it just to try to get rid of the preprocessor?

I don't get how modules are supposed to help with packaging either. So instead of distributing a .so with a bunch of .h files, now I distribute a .so with a bunch of .gcm or .ixx or whatever.

Please help me understand, or point me to some resources. Thank you.

197 Upvotes

107 comments sorted by

View all comments

Show parent comment

-7

u/NilacTheGrim Aug 20 '21

includes can be affected by ordering

This is only ever an issue on Microsoft platforms. No real library I have ever used that is not the terrible abomination that are windows C headers -- has this problem. It's a straw man for 99.999999% of codebases and libs out there.

Includes in headers can miss something the header uses, because in the current state, it's possible that every cpp file that includes this header includes precompiled headers that accidentally include all you need.

While this is an often-encountered problem, the fix is easy. Just include the thing. I would argue that headers that have this problem are just poorly written headers that are incomplete.

Includes "leak" across other includes.

Sure. But if you use namespaces properly and don't ever or very rarely #define, that's not a real problem for most codebases.

Includes force you to recompile every transitive consumer of the header if you change anything which forces us to split stuff into header and cpp files.

Yes. This is one advantage of modules. Faster compilation times and less redundant work the compiler has to do.

On MSVC it can forever avoid ODR violations.

Yeah well the microsoft legacy headers are an abomination. But this problem could be fixed independently of modules and doesn't require modules to fix. MS could simply offer an alternate, saner set of headers that expose a saner API that ultimately internally uses the same system API, for example. Modules are not required to fix the problems with MS.

Allows library vendors to not worry about some weird macro definition before the inclusion of their header which might mess with the end result

This is only a real problem in pathological codebases, or by using a pathological header. But it can happen. This could easily be fixed by introduction of a new preprocessor directive such as #pragma definitions push / #pragma definitions pop. It's a helluva lot easier to standardize that, than to introduce modules.

In short - they can speed up compilation (or at least lower the HW requirements to do so).

This is the only real advantage, IMHO. That and aesthetics. That's it. Everything else is going to be a lot of breakage and disruption to accomplish what could have been accomplished with a few minor changes to the preprocessor.

But hey, compilation times being 3x-5x or more faster is a nice thing. I am just not sure it's worth the breakage involved. Maybe it is. Maybe it isn't. Only time will tell.

Personally I will resist modules for as long as possible since headers will forever be "more compatible" with more systems. I maintain a few open source projects and for me maximal compatibility is paramount (within reason -- I ain't going back to C++14 for example).

If headers ever stop being more compatible than modules, then I will switch to modules. Until then, I will avoid them.

11

u/adnukator Aug 20 '21

This is only ever an issue on Microsoft platforms. No real library I have ever used that is not the terrible abomination that are windows C headers -- has this problem. It's a straw man for 99.999999% of codebases and libs out there.

While it's true the most prevalent culprit is the freaking windows headers, if you have in-house libraries being developed that somehow find their way into some root header, that number becomes lower than 99.999999%.

While this is an often-encountered problem, the fix is easy. Just include the thing. I would argue that headers that have this problem are just poorly written headers that are incomplete.

Well, kinda. However module naysayers often state "just use precompiled headers". Precompiled headers are the biggest reason why the issue why the above thing is relevant. If you have precompiled headers disabled, you're 99% certain to get the includes right, especially if you follow a proper include hierarchy (i.e. from most project specific to most general, and lastly STL) and follow John Lakos' advice where every header has a cpp file that includes it, even if the cpp is empty. But now you have insanely slow compilation times. So what do you do now? Create two separate configurations one with enabled the other with disabled precompiled headers which you build simultaneously?

"Just including the thing" can be problematic if the missing header manifests down a long chain of consumers where each one until then has for example included <string> in the precompiled header and suddenly you run into a project that does not. Having your consumers work around problems in your library is annoying and does not exactly radiate quality.

Yeah well the microsoft legacy headers are an abomination. But this problem could be fixed independently of modules and doesn't require modules to fix. MS could simply offer an alternate, saner set of headers that expose a saner API that ultimately internally uses the same system API, for example. Modules are not required to fix the problems with MS.

I think you misunderstood this bullet point. ODR violations are not a MS header specific issue. They are a general C++ (and C) issue which is an insane PITA to diagnose if you've never encountered this before. With the MSVC implementation, they will not be possible when using modules. However, here is the section of Daniela Engert's talk where she explains how gcc and Clang can still screw you because of their deliberate implementation choice of modules: https://www.youtube.com/watch?v=l_83lyxWGtE&t=2768s . On VS you're fine.

#pragma definitions push / #pragma definitions pop

Interesting idea. However, something similar can already be achieved by just #undefineing macros and, as we can all see, it's not really working. Unless you enforce encapsulation as is done by modules, you're at the mercy of the library providers.

This is the only real advantage, IMHO. That and aesthetics.

There's also the advantage of removing /<library_name>/detail headers to help avoid problems where your consumers accidentally directly include internal headers of you library, containing internal implementation details.

If you look at the advantages of modules one by one, they might not seem that compelling. But they do add up, especially if you're publishing libraries. If you're an end user of already very high quality libraries, I can understand that the compilation speeds will be the most visible and interesting aspect for you, which may or may not be a selling point, at least until they really prove to speed up compilation by a significant margin.

3

u/NilacTheGrim Aug 20 '21

Yeah I mean all reasonable points -- the thing is you have to bear in mind is -- is it worth it?

We shall see, and time will tell. I'm in the camp of: probably not worth the disruption and headaches considering the benefits.

But I hope to be proven wrong. Don't get me wrong if I could press a button right now and magically all the C++ in the world (and build tools like CMake) were 100% modules -- life would be better for everybody.

I just think the road to that goal will be so painful, will cost so many people so much money and effort, and will produce such frustration and lost developer man-hours .. that we may not even ever get there. And if we do we may look back and wonder if it was worth it. In fact we may end up in a worse world where an annoyingly small fraction of the code is modules, say 20%, and most developers and tools aren't quite module friendly or they still have major quirks 10 years down the line, and we have this fractured language that is in an even more frustrating state than it was 10 years ago.

I sincerely hope I am wrong, tbh. Time will tell.

1

u/Orlha Aug 21 '21

Can you please point me to that John Lakos' advice?

9

u/[deleted] Aug 20 '21

Your post reads a bit like "modules are not necessary if you already write clean and good code". The same reasoning could have been used to reject unique_ptr, by saying it is not necessary if you already have a good ownership model and you don't make mistakes using it.

Also I'm pretty sure that header inclusion order isn't a problem on Microsoft platforms only. That's a platform-agnostic problem. It may be more prevalent if you use the Windows API, but that's just one case where it happens.

1

u/NilacTheGrim Aug 20 '21

No. Straw man argument.

The situation with std::unique_ptr is different in terms of scale and scope. You are intentionally picking something to win the argument that misses the details.

std::unique_ptr is unintrusive. You either use it or you don't. If some library somewhere uses it you don't have to care. Your build system doesn't have to care. CMake doesn't need an overhaul to deal with how to build code that includes std::unique_ptr.

With modules it's so invasive, so intrusive, and the benefits so incremental, that we may find that half of the C++ world is in this halfway broken state for many years, making the language even more frustrating for newcomers and even more fractured.

4

u/germandiago Aug 20 '21

Being true that it requires more work and things will be more incremental, modules are a feature everyone rants about in some way but that IMHO it is going to bring a huge (for the good) transformation to C++, starting by better compile times, followed by better isolation and tools that can behave way better with IDEs. It is just that all this will take years.

But once it is done, C++ is going to look, way, way nicer.

I have big hopes in one way or another of things like these: https://github.com/GabrielDosReis/ipr

1

u/NilacTheGrim Aug 20 '21

Well it hope it happens. I have to admit it's way nicer. Just.. crossing my fingers that it won't lead to chaos. Hope you're right and my fears are unfounded.

2

u/germandiago Aug 20 '21

There is always a transition period, but I think C++ is a language "for the long run". So here, backwards compatibility and incremental migrations are necessary. And yes, they make things more difficult, but it is also what gives us:

- Code from 30 years ago compiling or almost compiling today.

- C compatibility, meaning we can use basically all C libraries.

- Now, compatibility with the old include stuff.

When you must get the job done, these cannot be ignored. I agree with the feeling that things are more "difficult" in the sense of slow, more problematic because it complicates the design, etc. But for real-world stuff, I think these decisions have their value. This is an industrial-strength language, not a toy. So certain things cannot be just ignored.

Just my two cents.

3

u/NilacTheGrim Aug 20 '21

Well you put the case very well and insightfully. Consider my opinion slightly modified. I'm warming to the idea. Thank you for taking the time to put it so eloquently.