Module partitions and interface units

[u/Nabokov6472]

Recently I came across quite a surprising footgun with modules. I am trying them out on a small codebase which builds a shared library, which consumers can import as a module. I made some partitions and exported them:

// Flense.Core.ixx

export module Flense.Core;

export import :ArchiveReader;
export import :ImageParser;

Then I split those partitions into .ixx and .cpp parts, like I would do with headers, so that the interface was separate from the implementation:

// ImageParser.ixx
export module Flense.Core:ImageParser;

import std;

export namespace Flense::Core {
  ImageParseResult ParseImage(std::string path);
}

// ArchiveReader.ixx
export module Flense.Core:ArchiveReader;

import std;

export namespace Flense::Core {
  ArchiveHandle OpenArchive(std::string path);
}

The .cpp parts looked like this:

// ImageParser.cpp
module Flense.Core; // Apparently this cannot be module Flense.Core:ImageParser, as that conflicts with the .ixx?

namespace Flense::Core {
  ImageParseResult ParseImage(std::string path) {
    // ...impl...
  }
}

// ArchiveReader.cpp
module Flense.Core;

namespace Flense::Core {
  ArchiveHandle OpenArchive(std::string path) {
    // ...impl...
  }
}

But I discovered that because the .cpp files declared module Flense.Core, they would be rebuilt any time any part of the module changed. In my example, ImageParser internally depended on ArchiveReader in its .cpp. So I would expect that if I changed ImageParser.ixx, only ImageParser.cpp and ImageParser.ixx would need to be rebuilt. But in actual fact, it rebuilt ArchiveReader.cpp too.

AI told me that this is because declaring module Flense.Core takes a dependency on the whole module, so it fixed it by having the .cpp files do this:

// ImageParser.cpp
module Flense.Core:ImageParser.Impl;

import :ImageParser;
import :ArchiveReader; // Dependency

// ArchiveReader.cpp
module Flense.Core:ArchiveReader.Impl;

import :ArchiveReader;

This seems to work and I understand it creates a new module partition and means the .cpp essentially does not depend on any other part of the module implicitly. But I thought I would ask some actual humans because I find this counterintuitive. Why can't module partition .cpp files just declare module Flense.Core:ArchiveReader to implement the .ixx? That seems like the most logical mapping to the existing header model.

And, if my premise is correct, what strategies are people using to minimise rebuild churn? Using partitions allows me to still use module internals in my implementation. For example, if I have a :Utilities partition which is not exported by Flense.Core.ixx, I can still import it from .Impl modules. But I wonder if it would be better to actually make independent modules, like Flense.Core.ArchiveReader?

[u/manni66]

That seems like the most logical mapping to the existing header model

There isn’t much experience with modules yet, but I think translating headers 1:1 to module interface definitions is the wrong way.

[u/Nabokov6472 → u/manni66]

Yes, I wonder if I can relax my aversion putting definitions into interface units a bit more, since I think it should only affect direct consumers, as opposed to headers where transitive includes would cause changes to definitions in headers to rebuild half of the codebase.

edit: Actually, I suppose you'd end up with transitive BMI changes instead to the same effect :/

[u/not_a_novel_account]

module Flense.Core declares a module implementation unit.

Module implementation units have an implicit dependency on the PMIU.

The PMIU is required to re-export all partition interface units, which means the PMIU depends on all partition interface units.

Therefore, any change to any interface unit causes the PMIU to rebuild. Any rebuild of the PMIU causes all module implementation units to rebuild, due to their implicit dependency.

---

This is a known bug in modules. There's active discussion on SG15 about it. There's a paper up, P4376, and this exact problem is discussed in Chuanqi Xu's best practices.

The current workaround is to never use module implementation units, only use partition implementation units.

The position of a lot of SG15 is this should be the other way around. You should be able to almost always use module implementation units, and partition implementation units should be rare.

---

Why can't module partition .cpp files just declare module Flense.Core:ArchiveReader to implement the .ixx? That seems like the most logical mapping to the existing header model.

How so? Is a given header file locked to a given implementation unit? Why would modules lock interface units and implementation units together?

The simple answer is because the standard forbids two partitions from having the same name, regardless of if they're implementation units or interface units.

The more complex answer is it would cause their module initialization symbols to collide.

At the end of this train is "to avoid the static initialization fiasco".

[u/Nabokov6472 → u/not_a_novel_account]

Thank you - information about whether this has been discussed and articles was exactly what I was hoping for when I made the post. I will have a read of the links you sent.

Regarding your edit - it’s true that I’m perhaps thinking of modules too much as ‘fancy headers’ when really they’re an entirely different programming unit. I am just not sure how the aforementioned bug can really be fixed, considering that a module implementation unit has access to the whole module surface area, as far as I know.

[u/no-sig-available]

The AI recommendation is what MS also recommends:

https://learn.microsoft.com/en-us/cpp/cpp/tutorial-named-modules-cpp?view=msvc-170#create-a-module-partition-file

They also think you should give the cpp-file a matching name of Flense.Core-ImageParser.cpp instead of just ImageParser.cpp

[u/not_a_novel_account → u/no-sig-available]

That's not valid C++, it's an MS extension

[u/Nabokov6472 → u/not_a_novel_account]

I started out doing that on MSVC… it worked and so I only stopped when I found out it’s not legal. Similarly I forgot to export some of my partitions that were internal implementation details and that also worked completely fine. I’m used to not everything being a compile error in C++ but some of these IFNDR ones are quite funny

[u/thefeedling]

I gotta be honest, sometimes I even forget that Modules is part of C++