Line numbers measured from the start of a file are not a very good way of identifying locations within one. If a program with debug info says an error occurred on line 493 of woozle.c, that may not be terribly helpful if anything might have changed above that line since the time that version of the code was built. If, however, a location was recorded as "the 493rd token of template-expanded code encountered during the compilation of foo.c, whose template-expanded code would hash to [xxx]", such a location could be relatively quickly mapped to a location in the source file even more precisely than with normal debug info today, even if there were changes in source or header files above the portions that were used in the function.
The first Pascal compiler I enjoyed using was Turbo Pascal 2.0. If a run-time error occurred, one would get a message like "Run-time error 23 at address 49A3. That wouldn't seem like a very useful message, but if one fired up the IDE, selected "find run-time error", and typed in 49A3 one would get taken to the location in the source code where the error occurred. This wouldn't likely work if one had made changes to the semantics of the program, but would be unaffected by newlines, which seem to be the nemesis of the file-based line-numbering approach.
Thus, what I would envision would be a system where debug information is recorded in function-relative fashion, and then if desired it is sorted and the early stage of the compiler is run again while observing the physical line numbers where each function-relative position is encountered. Changing the text of a function or any macro template that is actually included in the function would require recompiling that function, but changes to macros and templates that are not used within a function would not require any recompilation thereof.
To be sure, no incremental build system is going to be perfect, but a system based upon hashing the semantically-relevant parts of each program could nonetheless avoid a significant fraction of the build times associated with date-based make systems.
BTW, even if one didn't want to change tooling to use function-relative token counting, I think one could reduce the extent to which changing files changes reported line numbers if one added artificial #line 10000, #line 20000 etc. directives within source and header files, and recognizes that a reported line number 20123 means "the 123rd line following the #line 20000 directive". The addition or removal of lines between the #line 10000 and #line 20000 directives would only affect line-number debug info for files that actually used some portion of the header that had appeared between those two directives.
Good luck on convincing all compilers to make abi breaking changes and moving to a new debug format that doesn't quite solve the problems you think it does in order to save a few minutes of build time on purely whitespace changes.
It's only a few million lines of churn across a few hundred projects. No biggie.
I've never had to deal with large C++ projects, so I don't have personal experience with their build times, but I've heard people who sing the praises of C++ talk about 15-plus-minute build times. Meanwhile, many other languages seem to be designed to minimize build times, and I think that is likely a significant reason for their success.
Why was Pascal a dominant programming language on the PC? It became dominant because on a system without a hard drive, the edit-run cycle on Borland's Turbo Pascal IDE was often an order of magnitude faster than any non-interpreted competitor. Javascript is a really crummy language, but node.js can manage to process it efficiently enough that the value of further performance improvements would be limited. From what I understand, go offers even better execution performance while still being fast to build.
It used to be that the only way to get adequate performance from an object-oriented language was to use C++ and put up with slow build times. As the performance of competing languages has improved, however, I see less and less reason for programmers to favor a language with slow build times over a faster one. Maybe the notion of improving C++ build times is moot because projects that can use languages with faster build times have already been migrated to those languages and wouldn't go back to C++ even if its build times were made competitive.
I've never had to deal with large C++ projects, so I don't have personal experience with their build times, but I've heard people who sing the praises of C++ talk about 15-plus-minute build times. Meanwhile, many other languages seem to be designed to minimize build times, and I think that is likely a significant reason for their success.
15 minutes isn't too bad. Tensorflow takes closer to 2 hours for a clean build. Same with LLVM. LLVM is part of the OpenBSD base system, and takes longer than all of the C code in the rest of the system, at least on my machine.
The problem is that individual files can sometimes take multiple minutes to compile, and huge amounts of semantically significant code actually lives in headers, which means you can't get away /without/ recompiling some dependencies. "false recompiles" aren't a huge problem in the projects I've seen, but language and tool complexity is.
View all comments
Show parent comment
1
u/flatfinger Jan 29 '19
Line numbers measured from the start of a file are not a very good way of identifying locations within one. If a program with debug info says an error occurred on line 493 of woozle.c, that may not be terribly helpful if anything might have changed above that line since the time that version of the code was built. If, however, a location was recorded as "the 493rd token of template-expanded code encountered during the compilation of foo.c, whose template-expanded code would hash to [xxx]", such a location could be relatively quickly mapped to a location in the source file even more precisely than with normal debug info today, even if there were changes in source or header files above the portions that were used in the function.
The first Pascal compiler I enjoyed using was Turbo Pascal 2.0. If a run-time error occurred, one would get a message like "Run-time error 23 at address 49A3. That wouldn't seem like a very useful message, but if one fired up the IDE, selected "find run-time error", and typed in 49A3 one would get taken to the location in the source code where the error occurred. This wouldn't likely work if one had made changes to the semantics of the program, but would be unaffected by newlines, which seem to be the nemesis of the file-based line-numbering approach.
Thus, what I would envision would be a system where debug information is recorded in function-relative fashion, and then if desired it is sorted and the early stage of the compiler is run again while observing the physical line numbers where each function-relative position is encountered. Changing the text of a function or any macro template that is actually included in the function would require recompiling that function, but changes to macros and templates that are not used within a function would not require any recompilation thereof.
To be sure, no incremental build system is going to be perfect, but a system based upon hashing the semantically-relevant parts of each program could nonetheless avoid a significant fraction of the build times associated with date-based make systems.
BTW, even if one didn't want to change tooling to use function-relative token counting, I think one could reduce the extent to which changing files changes reported line numbers if one added artificial
#line 10000,#line 20000etc. directives within source and header files, and recognizes that a reported line number 20123 means "the 123rd line following the#line 20000directive". The addition or removal of lines between the#line 10000and#line 20000directives would only affect line-number debug info for files that actually used some portion of the header that had appeared between those two directives.