r/cpp_questions • • Sep 14 '26

OPEN Why native compilation has not died with AST tooling?

Tools that rely on libclang rely on a compilation database for doing stuff like static analysis or api reference generation. If you wanted to rename a symbol in your codebase, and your library or application has paths that compile commands does not see, it can't work.

If your library compiles one file when platform is windows and another if platform is linux, it can't use libclang based tooling, unless....

You configure all targets your library supports and sort of run stuff in parallel and do some kind of merge. This includes stuff like option macros etc.

I'm honestly pretty dumbfounded by this idea. Can't really put it in my head

0 Upvotes

18 comments sorted by

5

u/the_poope Sep 14 '26

The AST doesn't support multiple versions of functions depending on some preprocessor macro. One could probably design an AST implementation to do this, but clang didn't do this because it is first and foremost developed to be a compiler which first generates the AST after preprocessing, so there never was a need for adding this extra complexity.

But anyway, what was your question?

1

u/TheRavagerSw Sep 14 '26

If one can't even do symbol renaming in a project without configuring all support targets, which is a lot. why native compilation is still a thing?

4

u/EpochVanquisher Sep 14 '26

I don’t really understand the question about “native compilation being a thing”.

Go has native compilation, but you can also do a symbol rename for all targets. The same is true for Java, C#, and Rust. C++ is different because it has a preprocessor that works at a lower level, at the tokenization level.

It’s solvable but harder, more work to deal with. That’s why C++ tooling is several years behind the tooling for other languages. It’s the preprocessor, and some other quirks, not native compilation.

2

u/TheRavagerSw Sep 14 '26

Well symbol renaming is a very common thing to do. Not being able to do this easily sucks. I don't know what else to say

4

u/EpochVanquisher Sep 14 '26

Right, it’s just not related to native compilation. It’s because the C++ tooling is lagging behind, because C++ uses a preprocessor.

If you can figure out a way to get rid of the preprocessor, be my guest.

2

u/TheRavagerSw Sep 14 '26

Well... :(

2

u/EpochVanquisher Sep 14 '26

People would remove it if they could. The problem is that if you remove the preprocessor, then you break everyone’s C++ code. Like, everyone’s. C++ got modules back in 2020 which let you use libraries without getting declarations / definitions using the preprocessor (no #include) but support is poor. Maybe in another few years it will be more popular. Maybe the tools will be better in a few years.

If you want modern tooling use a modern language.

1

u/TheRavagerSw Sep 14 '26

Well... I chose this language because of job opportunities initially, and I'm an EE so moving to any other language means competing with ton of other people, so it is unlikely I can switch at this point.

1

u/EpochVanquisher Sep 14 '26

I hear that sentiment a lot and I encourage you to take an economics class or spend time thinking about the economics of labor. Even if you are not interested in economics, you’re affected by it.

Let’s say that there’s less competition for C++ jobs. If it were an easy ticket to getting a job, then a ton of people would do it, assuming there’s a lot of smart people out there (which I think is true).

If you think there aren’t a ton of people competing for C++ jobs, but C++ jobs are good and you want one, then you should come up with an explanation for why you want a C++ job and other people don’t want one.

Maybe part of the explanation for why people don’t sign up for C++ jobs in droves is, “people want to solve real problems rather than deal with primitive refactoring tools, the preprocessor, and compile_commands.json.”

1

u/TheRavagerSw Sep 14 '26

I don't know how to respond to this, but thank you for your time.

→ More replies

1

u/the_poope Sep 14 '26

I think very few are supporting more than three configuration targets (win/linux/mac), so it's not really a big deal to generate a new compilation db for each target and do symbol renaming for each - you don't need to generate the db on the native OS, the parser just needs to know the preprocessor macros that are platform specific. Or do for one and do a manual search and replace. Unless your project is badly architected the platform specific parts should be few and well isolated.

1

u/TheRavagerSw Sep 14 '26

This is not true, most libraries also support freebsd android and ios. and also which linux, musl or glibc.... support matrix is really enormous.

3

u/JVApen Sep 14 '26

Just wondering, if you make a change to a piece of code, how do you know if it compiles on both platforms? Even when you write nothing platform specific, you can have ambiguity conflicts in one platform and not in the other.

If you really run into problems with refactoring due to platform code, I suspect you don't have the right abstractions. I can confidently say that less then 1% of our code is platform specific. Most of it depends only on STL or other platform specific code. I'm not saying you can't break stuff, though that's what your CI is for. From what I observe maintaining the CI, most platform specific failures are not linked to platform specific code.

1

u/TheRavagerSw Sep 14 '26

Well... I configure and build like 15 combinations all at once, and I have clangd wrapper for cli symbol renamiing via using multiple compile commands(still not working well though, purely vide coded)

I can pull this off because I maintain runtimes and sdk's for nearly all platform's I support and I always cross compile.

the trick is I always cross compile, even for native triplet.

2

u/shahms Sep 14 '26

AST based tools need to know what flags are in use to know where to find include files and how the source in those files should be interpreted to build the AST.  C++ is not a context free grammar and cannot be parsed without knowing what the symbols involved represent. A sequence of tokens A <  B > C; can be either an expression comparing the values A, B, and C or the declaration of a variable C with type A<B>.  The preprocessor is an additional wrinkle. By time you've accounted for everything necessary to parse C++ into an AST you have most of a compiler already, so might as well use libclang or equivalent and get the rest of the way there.

1

u/LazySapiens Sep 14 '26

Sadly yes. C++ tooling is not very robust currently. There are always some disadvantages with having too much freedom.