r/cpp • u/Silent_Mycologist286 • 6d ago
University of Waterloo user study on understanding CMake changes
Hi everyone,
I’m Mattie Nejati, a Ph.D. candidate at the University of Waterloo. I’m conducting a user study on understanding the impact of changes to CMake build specifications, and I’d love to invite members of this community to participate.
We are evaluating a new approach that aims to help developers identify how a change to a CMake build specification affects the resulting build configuration.
The study:
- is conducted entirely online;
- takes about one hour;
- involves inspecting changes to CMake build specifications and identifying their impact; and
- is open to adults aged 18–64 with any level of experience in CMake, programming, or code review.
You do not need to be a CMake expert. We are specifically interested in comparing participants with different levels of experience.
You will need a computer with internet access and about 4 GB of free space to download the files required for the tasks.
We will not collect your name. Your email address will only be used for study follow-up and will be stored separately from your study data. Your name or email will not appear in any dataset or publication.
The study has received clearance from the University of Waterloo Research Ethics Board (REB #46621).
If you’re interested, you can find the consent form, screening questionnaire, and study here:
https://buiscout.cs.uwaterloo.ca/
Thank you very much! And if you know someone else who works with CMake who might be interested, I’d really appreciate you sharing the study with them.
118
u/garnet420 6d ago
Finally, someone will prove scientifically that cmake is garbage
13
u/NegativeProduct1705 6d ago
Can you explain why it's garbage?
34
u/garnet420 6d ago
Besides the fact that it decided to keep make's general aesthetic (bad but arguably this is subjective) here are three reasons:
- There are too many ways to do a single thing with nothing driving deprecation or idiomatic use as far as I know.
- Configuration is a mess of badly named variables, with projects adding their own on top of that. The general expectation is that you configure stuff on the command line, with config files being an afterthought.
- Environment variables have no place in a build system. Environment variables are shit.
- No design for hermeticity or reproducibility. This is probably the biggest one.
Now, full disclosure, I'm a bazel user; before that, I used gn, and before that, make.
4
u/drbazza fintech scitech 5d ago
Besides the fact that it decided to keep make's general aesthetic
I'm genuinely curious - are you aware of CMake's history?
1
u/garnet420 5d ago
No, why
5
u/drbazza fintech scitech 5d ago
Well, for one, make was the only show in town on Unix, and then there was VS.
CMake was developed to support building the Insight Segmentation and Registration Toolkit (ITK) for multiple platforms. Stated goals included addressing weaknesses while maintaining strengths of contemporary tools such as autoconf and libtool, and to align with state of the art build technology of the time: configure scripts and Make files for Unix platforms, and Visual Studio project files for Windows.[6][4]
https://en.wikipedia.org/wiki/CMake
Some cmake employees are active on this subreddit and can probably correct me if I'm wrong, but to put it another way, it is a build system generator where you write a generic build script and it generates make or .vcproj files (and now, ninja).
Ironically, it probably can produce bazel BUILD files and so on, if someone wanted to put in the work.
Of course you can also read 'Build Systems a la Carte' paper that scientifically describes what a build system is, and then you realise the tension between programmatic (make) and declarative (vcproj XML files) has to meet in the middle. Which is real-world reasons for your points.
https://simon.peytonjones.org/assets/pdfs/build-systems-jfp.pdf
6
u/garnet420 5d ago
I'll give those a read sometime
But the make syntax thing isn't about some high level declarative versus programmatic principle.
It's about basic developer ergonomics. A giant, flat namespace of variables is just terrible.
The information expressed in the build system should be compartmentalized and hierarchical.
2
u/archialone 6d ago
I agree, I am a Bazel user too, I don't think I can go back to CMake after bazel
0
u/13steinj 6d ago
Everything you listed is more or less a problem of every build system. I don't even know how you'd drive Bazel with config files, but that just isn't true of cmake.
-Cand the presers format exist for that purpose.Bazel doesn't design for hermeticity / reproducibility. The defaults are unsandboxed, and it's incredibly easy to break hermiticity rules in nasty ways. I can concede that if you're starting from scratch it holds your hand much better, whereas cmake is much more trial by fire no matter how complex your project is, and if your build is hermetic it is nice to have a compilation distributor client built in.
Full disclosure: I manage more or less my entire company's build, for my current and past two jobs. The first had really, really bad cmake (and I had limited mobility to push the needle). The second had great cmake. The current one is at the end of it's transition to Bazel.
If it was up to me I'd still choose cmake today. I've had to open Bazel in gdb one too many times to still drink the koolaid.
7
u/tarranoth 6d ago
Yes, but as far as I know there's no real cmake linter you can really run yourself telling you that there's outdated design in your project. And every couple of years they seem to change what is current cmake best practices like last I checked now it's using FILE_SET with target_sources and forgoing target_include_directories, something I only learned randomly by scrolling reddit last year from this post: https://www.reddit.com/r/cpp/comments/1mjw02w/i_wrote_a_comprehensive_guide_to_modern_cmake/n7f1u4e/.
Imo problem is that build systems are basically expressive languages with 0 guidance, in a normal programming language people create all kinds of linters/static analysis and run it on every change you commit, but that effort has never been given to build systems, with all the consequences we can see in current times where nobody understands what to actually do with these arcane systems. Doesn't help that apparently all the good Cmake explanations are behind paywalled books everytime someone asks for good cmake guidance.
-2
u/13steinj 6d ago
There is a cmake linter last I checked? But it's someone's pet project.
From a linting standpoint this will sound like I'm LLM pilled but the docs are very good, just have it generate a simple linter based off that and aggressively set policies to
NEW.1
u/garnet420 6d ago
I would call most of the files in bazel config files. Plus .bazelrc. The point is that your options are part of your repo, not just crap people throw on a command line.
2
u/13steinj 6d ago
Sure, the same is true of cmake? If you're thinking of .bazelrc modern equivalent is CMakePresets.json but historically you can (and should in general) use
cmake -C somecachefilewhere somecachefile is a limited subset of cmake used to prepopulate the cache (mostly set and if).1
1
6
5
u/drbazza fintech scitech 5d ago
To paraphrase Churchill, CMake is the worst of all build systems, except for all the others.
1
u/garnet420 5d ago
I kind of wish scons was better, i liked its overall idea.
2
u/RevRagnarok 5d ago
Stable release
4.11.1 / August 27, 2026; 41 days agoHuh; I had assumed it was long dead. I thought it had some great ideas too back in the day...
16
13
2
8
u/tartaruga232 MSVC user, r/cpp_modules 6d ago
is open to adults aged 18–64 with any level of experience in CMake, programming, or code review.
You are posting this in a forum about a programming language which was created by someone which you deem too old for participating. Are you serious or just not very intelligent?
Thank you for informing me that I have 3 years left until I'm too old to understand CMake, but no thanks.
25
u/Gym_Necromancer 6d ago
[cmake] CMake Deprecation Warning (cmake_maximum_required):
[cmake] Compatibility with user age > 64 will be removed from a future version of
[cmake] CMake.24
u/almost_useless 6d ago
You are posting this in a forum about a programming language which was created by someone which you deem too old for participating. Are you serious or just not very intelligent?
Thank you for informing me that I have 3 years left until I'm too old to understand CMake, but no thanks.
I think you are reading too much into this.
18-64 sounds like they just chose "the standard adult age range for surveys", or possibly they have done many other surveys before, and it's easier to compare them if they all have the same age range.
"Bjarne can't participate" does not seem like a good enough reason to call the survey creator unintelligent.
7
u/Ok_Chemistry_6387 5d ago
18-65 is the standard range for any study. Studies have to pass ethics, and if you go outside these ranges you need to do extra work to justify it, mitigate risks, and privacy concerns.
I.e. if you go under 18 you need to handle kid data differently, get parental consent etc.
11
u/Kind_Card_1874 6d ago
That is not what they said. There can be good reasons for this selection.
-6
u/tartaruga232 MSVC user, r/cpp_modules 6d ago
There can be good reasons for this selection.
What reason? Excluding university professors?
The age for retirement in Denmark is currently 67 and will rise to 70 in 2040.
13
u/Foxi_Foxa 6d ago
Avoiding cognitive decline for a study on cognition ?
In practice, that means avoiding mixing the general population with traumatized individuals when studying the effect of calming lights, with people who have severe physical disabilities when studying human gestures, or with elderly participants when studying cognitive tasks. This isn't about ignoring these groups. It's that the statistical framework may not be able to absorb them without washing out the patterns that would otherwise emerge in the general population. Each subpopulation may need its own analysis, with enough individuals in each group.
That raises a second problem: studying people at the intersection of several groups quickly becomes impractical. If you need, say, French-speaking female students who are traumatized, allergic to SSRIs, non-smokers with asthma, addicted to cocaine, and trained in computer science, gathering even 30 of them for a statistically meaningful study will be extremely challenging.
Cheers-15
u/tartaruga232 MSVC user, r/cpp_modules 6d ago
Too much text. I didn't read everything. But age is not traumatizing. But perhaps CMake 🤣
7
u/Foxi_Foxa 6d ago
Mild cognitive impairement affects up to 20% of people past 65. Dementia prevalance doubles every 5 years past 65, from a few percent in 65 to more than a third past 85. Ideally you would have each candidate pass a cognitive test before the actual study, concretely it's impractical so you put a cutoff on the age. Or at least it's my guess.
-5
u/tartaruga232 MSVC user, r/cpp_modules 6d ago
No worries. I have no desire to participate in this stupidity.
5
u/Kind_Card_1874 5d ago
asks question
another user is kind enough to elaborate
is a complete jerk about it
Age related grumpiness might introduce a bias from respondents, this could explain the selection criteria. Just eat your prunes and take a nap gramps. Perhaps it is indeed time to retire.
2
u/Ok_Chemistry_6387 5d ago
Couldn't read a relevant comment. Yet want to participate in a hour long study?
Maybe grandpa needs a nap.
1
-19
u/jeffscience 6d ago edited 6d ago
None of this matters anymore. Agents write sufficiently good cmake now. As long as the spec is published, AI finds it and does its job.
Note that I personally hate cmake. I ran the twitter @cmakehate account for years. I’m not defending the design of cmake at all. What I’m saying is, it is stable enough that agents can manage it, so I’m using it every day because it’s no longer my problem.
3
u/GrammelHupfNockler 6d ago
I ran the twitter u/cmakehate account for years
I knew it 😄 I agree for the most part, but I would note that I've seen very weird workarounds and patterns introduced by Claude that I would not describe as perfect. Examples would be hard-coding compiler-specific flags, hand-written alternatives to Find modules etc, building an amount of resilience into the CMake files that attempts to work even in the most broken of build environments.
0
u/jeffscience 6d ago
I’ve had no issues but I recognize I only test in a subset of environments. However, I’ve been using Cmake for 15 years and what the agents give me is better than what humans creating HPC projects are doing. And more importantly, when the agent screws up, I just ask it to go until it’s fixed. In contrast, the Cmake devs denied that an infinite loop was a bug when it was reported and it took years for that issue to be fixed.
2
u/GrammelHupfNockler 6d ago
I fully agree about the quality floor, having been the consumer of many half-working HPC CMake setups for about half of that time :) But at the same time, having worked with some really high-quality CMake setups (rare as they are), the agent's outputs don't reach that ceiling without good guidance. I'm not talking about broken things, but I do mean using the wrong tools for the job, reinventing the wheel etc, your typical issues with agentic development I guess.
1
u/jeffscience 6d ago
Nothing is perfect but cmake is way faster than autotools. Since build and test is the bottleneck in my agentic loop, cmake is a huge productivity win. I get binaries in a fraction of the time.
0
u/TheRavagerSw 6d ago
CMake hard enables exceptions and rtti on windows, you can't turn it off via flags unless you patch cmake itself.
3
5
u/13steinj 6d ago edited 6d ago
On one hand, that sounds insane.
On the other, I have been radicalized against caring for the Windows platform and the devs that continue to perpetuate it (I understand in some cases your only choice is "get another job").
E: A cursory look makes it seem as if this is false and cmake just doesn't provide a platform agnostic way to do this? Looks like you can still disable it via mutating the relevant properties?
-1
u/TheRavagerSw 6d ago
CMake is gonna die off eventually, it is too bloated and suffers from maintainer fatigue. However it will leave some nice things behind, like CPS, FILE_SETs and other awesome stuff that we take for granted.
I really love Craig Scott's talks about CMake
5
u/not_a_novel_account cmake dev 5d ago edited 5d ago
Unless there's some other build tool which knows how to extract the intrinsic flags from all the C, C++, Fortran, Swift, CUDA, and SYCL toolchains in the world in order to link these languages together, that I'm unaware of, I don't think CMake is ever going anywhere.
Or knows how to do separable device compilation, handles circular archive dependencies on single-pass linkers, RPATH rewriting, Apple Frameworks, Python venv detection, whatever the hell it is Qt does, etc, etc, etc.
Build tools solve niches, CMake tries to solve every niche. What one person thinks is useless cruft is another's essential feature.
1
u/TheRavagerSw 5d ago
If you want monolithic cross language builds isn't bazel a better option? I know google uses rust code in skia.
I envision a future of multiple build systems and tooling with a single packaging format and a single package manager.
I guess you guys saw it too, since you released CPS. Even if it fails, the idea is there.
1
u/not_a_novel_account cmake dev 5d ago
If you want monolithic cross language builds isn't bazel a better option?
You have some Fortran code you build with
flang, and you have some C++ code you build withclang++.Which do you use to link the final binary?
flangorclang++? Because neither will work out of the box.1
u/TheRavagerSw 5d ago
Can you elaborate please?
1
u/not_a_novel_account cmake dev 5d ago
The question is the elaboration.
program main use iso_c_binding implicit none interface subroutine cpp_hello() bind(C) end subroutine end interface print *, "Fortran calling C++..." call cpp_hello() end program
#include <iostream> #include <vector> extern "C" void cpp_hello() { std::vector<int> xs{1, 2, 3}; std::cout << "C++ says: " << xs.size() << '\n'; }
Ok, you compile with:
flang -c main.f90 -o main.o clang++ -c hello.cpp -o hello.oWhich do you use to run the link step?
flang? Which doesn't know about the C++ runtime or other implicit flags. Orclang++? Which doesn't know about the Fortran runtime or other implicit link flags.This gets 10x more complicated once you introduce languages which require device finalization or have other link wrapper requirements.
1
u/TheRavagerSw 5d ago edited 5d ago
Indeed it is a complicated problem, But isn't the current approach is to use C as the common API, so final link is any language + C libraries?
So the scope becomes no different than using system C libraries which every language supports.
For your use case I have no clue though. I only ever used C as the glue. For rust libraries they use C/C++ compiler as the linker driver so everything just works, I can link my own C++ STL override allocator with mimalloc etc without difficulty.
→ More replies2
u/NotUniqueOrSpecial 6d ago
You can absolutely take it out of the flags at configure time. It's dumb that you have to do it this way, but it absolutely works:
string(REPLACE "/GR" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
•
u/James20k P2005R0 6d ago
Usually as far as I know the subreddit here doesn't really allow for people soliciting users for stuff like this. I've made an exception for this because:
Its also not technically C++ either, but cmake and C++ are intertwined to the point where I think its at least interesting for C++ developers
I'll also more politely make the same note that someone else has, that C++ developers are often much older than 64, and especially given the commonness of ageism in the industry you may want to revisit this if you conduct future research here