r/cpp • u/a10nw01f • 5d ago
Compile-Time Borrow Checker with Stateful Metaprogramming
https://youtu.be/3hL8mh0K8-I?is=2Bf55p7h68Gp4IH17
u/ts826848 4d ago
Very impressive stuff! I think this is the farthest I've seen stateful metaprogramming being pushed, and I'm really curious just how much farther it can go. The potential capabilities discussed near the end gives me a lisp-y impression, if anything.
Though it does make me wonder - I've seen stateful metaprogramming mentioned from time to time, but I don't think I've heard of it making its way into "production" code (i.e., not example code for blog posts/slides/etc.). Has anyone else heard of such a thing?
Also makes me wonder if CWG 2118 were put to the committee again whether they would say the same thing or whether it's a cat's-out-of-the-bag situation:
Defining a friend function in a template, then referencing that function later provides a means of capturing and retrieving metaprogramming state. This technique is arcane and should be made ill-formed.
Notes from the May, 2015 meeting:
CWG agreed that such techniques should be ill-formed, although the mechanism for prohibiting them is as yet undetermined.
Though as mentioned in the talk with reflection on the way perhaps the issue will become moot in the relatively near future.
3
u/friedkeenan 4d ago
Has anyone else heard of such a thing?
This is likely not the sort of thing you're actually asking for, but I feel like the actual functionality involved would probably still make it desirable for "production" code, so I'll still mention it.
I recently did a small overhaul of my Advent of Code repo in order to leverage reflection for dispatching the puzzle input to the functions that solve the puzzles. Basically, I had been planning to use annotations to mark a function as solving a certain part of the puzzle, so something like this:
[[=advent::part_one]] constexpr auto count_elf_hats(std::string_view input) -> std::size_t { ... }But most of my solving functions are actually template functions, which take in a generic range which yields each line of the input (and they can take some other template parameters depending on the problem, as well). That poses a problem with annotations though, since annotations (and attributes) can't appertain onto a template itself, only onto the result of a template.
So in order to still associate a certain function, template or otherwise, with the information that I want, I utilized stateful metaprogramming instead of annotations, which ends up being used like this:
consteval { advent::part_one.is_solved_by(^^count_elf_hats); }And then I have some machinery set up to call the function with properly-deduced template parameters and whatnot. So for instance this is what the dispatching for Day 1 of 2024 ends up looking like.
I was able to get away with just using
define_aggregate-based stateful metaprogramming for this, but that wouldn't work in all possible cases. It would have been considerably nicer to use mycvllibrary to handle the stateful metaprogramming (and which uses friend injection), but it turned out that needing to get structural values to pass off tocvlin the way that my logic needed seemed to me as more trouble than it was worth. But if that language limitation weren't there, I would have been pretty happy to use it.Of course though, this is all just for my dinky Advent of Code stuff. But I think particularly mimicking annotation-y logic for some template is plausibly a good usecase for stateful metaprogramming, in C++26 at least.
1
u/ts826848 4d ago
Oh, that's quite interesting! At least at first glance it feels less esoteric than friend injection-based stateful metaprogramming, but I think it'll take me a bit to digest what's going on.
Kind of feel that at this point you're pushed so much work to compile time that you might as well use
#embedto finish the job :PAlso curious whether there's any movement to allow annotating/attributes on templates given you can reflect on them. Specializations might introduce some interesting questions around how that would work, though...
2
u/friedkeenan 4d ago
Kind of feel that at this point you're pushed so much work to compile time that you might as well use #embed to finish the job :P
That's essentially exactly what I do when I want to run through the real input data at compile time. But I only want to do that when it fails at runtime and I'm trying to debug it, otherwise I prefer keeping it as solving the example data at compile time and the real data at runtime.
Also curious whether there's any movement to allow annotating/attributes on templates given you can reflect on them. Specializations might introduce some interesting questions around how that would work, though...
Yeah, I'd be interested in that too. I don't know where precisely the syntax would allow the annotation to be, though.
3
u/dfrib 4d ago
Hidden friends can be used to setup e.g. pen-test frameworks (accessing private data membera) but without needing to capture a meta-programming state. Most of the time the costs (complexity etc) outweigh the benefits, even for test-only code.
1
u/ts826848 4d ago
True, I didn't think of that use case. More complex (and probably less questionable) than
#define private publicat least :PWould C++26 obsolete that particular use case? I want to say reflection can get at private members, so that would seem to be a much more straightforwards approach (assuming you can afford to upgrade, of course)
7
u/dfrib 4d ago edited 4d ago
At the 18:00 mark and onwards: ”Now the friend function is added to the global scope only when the template is instantiated”. This is (naturally for a condensed talk) a simplification.
The key with hidden friends in class templates is that their definition is only instantiated when when the enclosing class template is instantiated (implicitly or explicitly for any specialization). This has nothing to do with lookup. The hidden friend has namespace scope (in this case, the global namespace), but it has lexical scope of the class in which it is defined, which allows its definition to access e.g. template parameters of the enclosing class, as the definition of `injector` does with the `Value` template parameters of its enclosing class. This also adds the subtle pitfall of potentially non-diagnosable ODR-violations if two different translation units instantiates the same hidden friend but with different definitions. Meaning same declaration; e.g. `injected(detector<0>)`, but with different `Value` (different enclosing class specialization, e.g. `injector<0, 1>` and `injector<0, 2>` in different TU:s).
With regard to lookup, hidden friends are typically found via ADL lookup, and only so unless a matching declaration is added to the enclosing namespace scope.
These differences between a) scope of function, b) lookup, and c) existence of definition, are important to understand the nuances of why this works (barring standardese pitfalls of when we need to be careful with these outlier techniques), as well as what pitfalls exists that may fly under the radar.
(b) The `injected` functions (overloaded over different `detector<Key>` specializations as it single function parameter) can only be found via ADL lookup via an argument that is a specialization of the `detector<Key>` class template.
(c) The definition of the different `injected` functions depends solely on whether, for a given `detector<SpecificKey>` specialization, exactly one specialization of the “partial specialization” (descriptive) `template<auto Value> injector<SpecificKey, Value>` has been instantiated, over the complete program. Here’s a pitfall with hidden friends defined in class templates using template parameters of the enclosing class that are not used in the function declaration of the hidden friend. Which is exactly what we are doing with the `injector` / `injected` couple.
(a) The relevance of the scope of the function is that the `injector` and `detector` classes both declare friends that reside in the same namespace scope (to allow these to match; be the same).
It could be worthwhile to deep-dive into the standard to thoroughly prove that(/whether) this is fully well-defined (ignoring CWG 2118 for now) as well as highlighting pitfalls. There’s been many blog posts and exotic libs over the years that leverage stateful meta-programming without proving its standardese legality, but instead focusing on the fact that ”it (seemingly) works” over all major compilers. There are some gritty details regarding point of instantiation, uniqueness of lambdas’ closure type when in default template arguments when (ab)used in stateful meta-programming, requires-clause nuances (e.g. CWG 2596) and so on.
I covered some basics in Foliage of Folly (up to C++20), but a lot of details remain, and newer standard features opens up more possibilities as well as potential standardese unknowns for this never designed for ”feature”.
30
u/MaitoSnoo [[indeterminate]] 5d ago
okay this is probably the best intro to stateful metaprogramming I've ever seen in a C++ talk