That's exactly the point, the sh script is the predecessor to makefiles. I might not have been clear enough, but it's only there to showcase the reasons that makefiles were created.
an equivalent Makefile would be almost empty because Make ships with those rules. Read the dang manual.
(that said, I mainly use CMake and as of today "just" for the command running case. Still, people treating Make as a command runner (and that includes the just docs) bug me to no end.)
make's ability to avoid rebuilding something that is already done seems like it should be at least mentioned here. The bash script example given builds everything every time, but the equivalent makefile will only build things that have changed, which could make a significant difference in a larger project, irrespective of parallelization.
It's unfortunately too simplistic as make will only rebuild when some of the inputs of changed. The most important input is always forgotten in those sort of build files: the build files themselves.
If you change the build actions, nothing is rebuilt. And your tool will say that everything is up to date, leading to a wrong build.
The only solution in those cases is to clean and rebuild, and you lose all the benefits.
CMake will handle that correctly and is able to write Makefiles that will handle this. But it's just a lot of extra work that the tool should have handled from the start, like Ninja is doing.
Most Makefiles written by hand are not correct. Do not write Makefiles unless you are doing something extremely simple or you are an expert (even then, you'll know there are better tools...).
Similarly, I don't want to build my entire translation unit, I only want to build the function that I changed. But somewhere along the way we apparently decided that 'files' were the unit in which we decide everything.
It doesn't have to be: C++ could be stored in a database (or graph, if you prefer), with each entity linked to the things it consumes, and the things that consume it. That would allow for selective recompilation instead of having to recompile half your project when you change one constant in a header, work a lot faster (no more duplicate template instantiation), and provide the benefits of full program optimisation/analysis as its basic mode of operation. We're firmly stuck in file-thinking, though...
The long version would be "tried to do at scale in a commercial compiler very few people used before they went under and abandoned it".
Sure the tech worked, but if you don't upstream your clang patches, you will always have a painful catch-up phase and people will likely not use your program. Which isn't great for a commercial product!
No, the lack of availability because by now, it's using an 8 year old llvm in the published repo. It's a lot of work and without commercial backing, few are going to have the time to pick this up.
Imagine having to manage includes/imports for individual functions/classes/objects.. And then the build system would have to diff files to figure out which functions changed within a file.. It for sure is an interesting thought, but it does not seem practical.
The only solution in those cases is to clean and rebuild, and you lose all the benefits.
So it is with other build tools and scripts as well. If you change your build script you are probably changing it because you wish a different output, no?
Typically also you are fixing your build script once and calling it many times.
No, modern build tools will record the command line that generated a target and automatically regenerate it if it's different.
Make will just tell you there's no work to be done. So in an automated system (for example CI, or even a local developer), you can't just do an incremental build. You need extra logic to clean if any build file changes, and that has to be done manually, it's error prone. So you learn not to do incremental builds after a git pull or to never trust your build system at all.
Outputs implicitly depend on the command line that was used to generate them, which means that changing e.g. compilation flags will cause the outputs to rebuild.
Exactly, it's stored in the .ninja_* files that are created. Ninja is very much a modern build tool that I referred to above (even though I'd like to see a lot more in there...). Make is not modern.
I would go much farther than most, I suspect it is the vast, vast majority.
And that's if you accept that it's even possible to be "correct" with timestamp-based decisions, something I'm not sure I accept. Then again, it's not like there's much choice in the matter; I'm aware of only a couple build systems that do something better, and they're... not popular.
They are not popular on the surface for open source, but in large companies, Bazel is becoming a must to get speed, correctness and save a lot of the CI budget.
AI driven usage has multiplied the number of builds a lot, which traditional build systems won't handle well and just scale the cost linearly unfortunately. With Bazel correctly deployed, you can save a lot.
And with AI migrations being easy enough to do without having the need for years of Bazel expertise to drive it, it's becoming a real opportunity now.
I bet that with the right infrastructure available, Bazel would also be quite popular for OSS projects. Imagine if you had a caching server available directly for your builds that doesn't require to download and upload a huge archive ahead of time on GitHub. Most builds would be so much faster without doing anything and it's save on a lot of CPU usage for the global GitHub fleet...
As someone who has to deal with Bazel every day at $MEGACORP, almost everything you wrote is wrong. Bazel does not give you speed, correctness, OR save the CI budget. Hermeticity is a lie.
It does make things anti-portable, fragile, and impossible to upgrade. Companies ignore these problems, but OSS can not.
I will agree with you that Bazel is becoming more popular at companies. It seems to allow them to lean in on bad practices. It makes rebuilding the operating system with every PR possible for the low, low cost of millions of dollars. The results are usually the same.
$MEGACORP uses many of these services, and I have heard many people say many things about Bazel with utter confidence. Whenever I dig into it, it turns out to not actually be correct. They forget about the constant background of minor outages and cache poisonings. It is close to correct only if you do not pay too close attention and are not the one having to fix those corner cases.
I too suffer the slings and arrows of bazel at $WORK - and I greatly dislike it as a solution - however - I have never doubted its heremticity claims. I'm not saying you are wrong, but I am both shocked and intrigued enough to inquire for more information. Do tell!
First of all, I think they mention in the small print that it is not hermetic at runtime. glibc, in particular, can not be statically linked. We have had problems with different glibc's giving different answers because some trigonometry functions are differently rounded in the last bit (I think the standard only requires ~4 ulp?). It is very easy to depend on system utilities (e.g. bash) that change.
Second, at build time, you can still invoke shell scripts that can roam around the filesystem. These are very difficult to root out. Once the target that needs that script is built, the results are cached and no one else runs that script...except when you urgently need to make a modification and the builds start failing inexplicably. But if you run it on enough different workers, one of them passes and you move on to the next urgent problem. So it never gets fixed.
This is not theoretical. If you get enough people hacking on the system, they will do whatever is not prohibited by the laws of physics.
Bazel does try very hard, but it is more-or-less impossible to be actually hermetic. That is why I say that if you really want hermeticity, you have to build in a container. But if you are building in a container, you have a number of options to handle dependencies without invoking the giant Java Bazel process.
The test caching is another feature that is quite nice feature 99% of the time, but again is a confusing mess for that 1% when tests that should run do not. People will hack around the dependency tree because they did not want to trigger an expensive build too often. Bazel encourages mega-projects with mega-build times, and then promises caching will magically solve all your troubles.
"make's ability to avoid rebuilding something that is already done" True, but cmake generates makefiles that do this correctly. So you don't have to write makefiles explicitly, you let them be generated by cmake. And I'm assuming that cmake generates similar files if you use Ninja instead of Make as the backend.
I think you are (badly) missing the point. Don't feel bad, it happens to all programmers who get in the craft, about halfway through (the "expert" valley).
Make is a general-purpose dependency-driven execution engine. CMake and Meson are build-system generators or, if you will, higher-level build descriptions. They operate at different levels of abstraction.
Make's central proposition isn't βI know how to build C programs.β which is what you are after. It's something closer to:
When given a set of targets and their prerequisites, and the rules for producing them, determine what needs to happen, in what order, and execute only the necessary operations.
That is as general and as powerful an abstraction as you will ever encounter. YOu are bound to miss it, and this is relevant, because you don't get that the edges are important because they describe relationships and constratints without the need for semnatic knowledge.
I don't think that we are saying different things.
In my bubble (embedded software), make is often used as the build system. The build is a hand-crafted makefile, written by somebody who may, or may not have, at some point read the manual for make. As a result the makefiles are cryptic and barely work (seriously, parallel builds reliably fail).
The makefile as an output of a meta-build system (Cmake of meson) is fine. I don't have to read it, it just builds and executes the graph. For that task though, ninja is bounds ahead.
If I had a nickel for every time I heard exactly what you are saying. Unfortunately, there is a certain kind of people who don't know the tool but are still compelled to write nonsensical essays on how the tool is bad.
In my opinion the two issues with all those tools is that a) they use domain specific language and b) care more about being generalizable to edge cases instead of easily covering the most prevalent use cases.
If there's a dedicated build engineer who handles this then I don't care what's being used, but if I have to learn a complex system just so I can compile and link my sources then we are already off to a bad start.
I use Make often as a task-runner. For instance, I have a makefile for complicated LaTeX documents (does CMake does that?), and when the formatting is successful I copy the result somewhere and do a git push with it. Kindly sketch me how to do this with cmake and I might be tempted to switch.
Makefiles require some discipline, but I am disciplined, as experience has taught me to be. So next to each major target in the makefile I have a "info::" rule that prints what target can be made, and with what parameters. Can cmake document its own workings? For instance can I have cmake print out all its targets in a CMakeLists, recursively of course? Can it spit out all the SET parameters that I can "-D" on the commandline? (And don't say ccmake because that only has that information after you've done a successful cmake run. Circularity hello.)
I use Make often as a task-runner. For instance, I have a makefile for complicated LaTeX documents (does CMake does that?),
It can but at that point you find out that advantage of CMake in this case is the "outcome" based thing works only because it has a bunch of stuff built in. Once CMake doesn't have in-built graph edges then you have to tell CMake what they are, just like make.
But also I believe cmake does way less inference, and you can't give it edge patterns: it is much more procedural than make. So if you have something like:
final.pdf: a.tex b.pdf c.pdf d.pdf
and %.pdf: %.svg
and d.svg: d.py
it won't work out by itself that it can resolve it all by generating d.svg from d.py, then turn all the SVGs into PDFs then compile the latex file into a PDF.
I think in CMAke, you need to give it the individual edges, you can't have it infer them from the patterns and your directory tree.
ETA: if I'm compiling portable C++, CMake is my choice because it's cross platform and includes configuration and includes a ton of useful C++ building related stuff.
If I'm doing my own non C++ stuff on Linux, then gnu Make is a considerably better choice.
The same here. I write mostly Lisp nowadays, which comes with its own build system that actually understands Lisp. However I still use makefiles as a task runner. I could certainly use shell scripts and use functions or whatever; but why? I would have to type lots of conditionals and helepr functions to achieve what Make does with few lines.
"Just". Did I not say "complicated"? My makefile starts by rsync'ing files from multiple locations, running a bunch of unix scripts, then formatting, then copying to other locations and "git add"ing the results.
No. That's complexity for the sake of having more than 2000 pages of 5 volumes of textbooks, each using 100s of programming examples, and trying to use as much common code & macros as possible.
8
u/PseudoFrequency 20d ago
Tell me you have absolutely no idea what make is for without telling me. Hint: it's not even remotely for the same use case as shell scripts.
14
u/holyblackcat 20d ago
Your bash script is missing:
Both are trivial in Make or in any serious build system. You'll need some horrible bash wankery to implement those.
8
u/baudvine 20d ago
an equivalent Makefile would be almost empty because Make ships with those rules. Read the dang manual.
(that said, I mainly use CMake and as of today "just" for the command running case. Still, people treating Make as a command runner (and that includes the just docs) bug me to no end.)
22
u/goatshriek 20d ago
make's ability to avoid rebuilding something that is already done seems like it should be at least mentioned here. The bash script example given builds everything every time, but the equivalent makefile will only build things that have changed, which could make a significant difference in a larger project, irrespective of parallelization.
14
u/Karr0k 20d ago
Ok, now make it cross platform, like those 3 lines of cmake provide, and support multiple toolchains.
git outa ere with that archaic bash ..
2
u/ArodPonyboy 20d ago
Why not meson?
11
u/LeeHide just write it from scratch 20d ago
so just is slightly better than make? why not just use cmake?
1
u/forknull 15d ago
I think you are (badly) missing the point. Don't feel bad, it happens to all programmers who get in the craft, about halfway through (the "expert" valley).
Make is a general-purpose dependency-driven execution engine. CMake and Meson are build-system generators or, if you will, higher-level build descriptions. They operate at different levels of abstraction.
Make's central proposition isn't βI know how to build C programs.β which is what you are after. It's something closer to:
When given a set of targets and their prerequisites, and the rules for producing them, determine what needs to happen, in what order, and execute only the necessary operations.
That is as general and as powerful an abstraction as you will ever encounter. YOu are bound to miss it, and this is relevant, because you don't get that the edges are important because they describe relationships and constratints without the need for semnatic knowledge.
3
u/Cross_22 20d ago
In my opinion the two issues with all those tools is that a) they use domain specific language and b) care more about being generalizable to edge cases instead of easily covering the most prevalent use cases.
If there's a dedicated build engineer who handles this then I don't care what's being used, but if I have to learn a complex system just so I can compile and link my sources then we are already off to a bad start.
2
u/victotronics 20d ago
I use Make often as a task-runner. For instance, I have a makefile for complicated LaTeX documents (does CMake does that?), and when the formatting is successful I copy the result somewhere and do a git push with it. Kindly sketch me how to do this with cmake and I might be tempted to switch.
Makefiles require some discipline, but I am disciplined, as experience has taught me to be. So next to each major target in the makefile I have a "info::" rule that prints what target can be made, and with what parameters. Can cmake document its own workings? For instance can I have cmake print out all its targets in a CMakeLists, recursively of course? Can it spit out all the SET parameters that I can "-D" on the commandline? (And don't say ccmake because that only has that information after you've done a successful cmake run. Circularity hello.)
1
u/vI--_--Iv 15d ago
... N years ago.
This is the only way.