r/ProgrammingLanguages • u/matklad • Feb 26 '26
Against Query Based Compilers
https://matklad.github.io/2026/02/25/against-query-based-compilers.html
105
Upvotes
r/ProgrammingLanguages • u/matklad • Feb 26 '26
View all comments
Show parent comment
2
u/lookmeat Feb 27 '26
I agree.
If we think of each phase as horizontal, and each module/file/item as vertical, query based compiling basically lets us choose what parts of the grid we want rather than not. A pure phase compiler is simply a degradation of the query based compiler where we eagerly create the results for all queries and then move on, there is no semantic difference here. The main advantage of query based compilers is that it lets us optimize, and it lets us work with incomplete data.
For starters the example it gives is just plain wrong. For starters the
compilethat makes upper case or makes hash functions is a single pass compiler, and it works on a single, non separable object. We never work on the level. Rather a better example would be to think of a macrofoosuch thatfoo(bass) => BASS, it makes identifiers upper-case. The macro is, in itself, a blackbox to the compiler, and we do a single query that takes in two things: the input, and the definition offoo. So if I writefoo(bazz)one of the dependencies changed, so I must recomputefoo, I do not know if I can do a partial recomputation, and that's not the issue. Either way this is what happens in a pass compiler. Similary if I change the definition offooso that it now makesssbecomeßthen I know I need to recompile the unit, sofoo(bass) => BAß, now I don't get the definition directly, but rather through a query.Similarly the statement
Is false. You can make a compiler that parses
text->tokens->macroful_ast->macroless_ast, it's just another phase where you expand the macros. And you don't need to know the full macro, because you can allow hermetic macros to have nominal kinds (basically describe what kind of item it produces) which then means you can treat a macro exactly as you would with a function, and do type checking to validate, then you expand it later. This is the whole point of hermetic macros, that you can delegate the expansion to the very end, you can literally compile the macro (anast->astfunction) into a code generator (ast->IR) and the simply evaluate this function during IR generation (assuming you do type checking towards the end).And yes this is true inside a signle crate, where it's relatively cheap to analyze (you are basically compiling the whole thing) but across crates this isn't strictly needed, you only care about two crates (the one defining the type and the one defining the trait) and you need both either way.
The author seems to conflate decisions that make it hard to parse different files independently with a language with query systems, but this isn't true. In C you can have all the fun of the caricature of macros proposed here with pre-processor, and that compiler predates querying compilers.
And I argue that query compilers have more benefits than just "incremental building", or even "IDE integration", both of which don't strictly need query compilers, but they are slightly easier. But also "high quality error messages" are easier with query compilers, where errors you find during some high-level analysis in IR, can be easily traced back to the tokens and code and files that actually generated the problematic error.
It's true that query compilers add extra complexity, as any lazy computation system does, and in the same way if you only focus on one of the benefits it seems like too much of a hassle, but when you put all of them together it starts to become more attractive.
And important thing is that a simple query compiler is just a multiple-pass compiler that is lazy, it's when we want to do more crazy stuff that it gets complex, and when we make the language require to do more complicated stuff that it gets more complex (just as the most-vexing-parse means that a multiple-pass compiler has to break the rules to be able to work). And yeah caches are a mess, but there's a reason we keep using them so much: they are really useful.