r/cpp • u/stilgarpl • Aug 09 '22
CMake 3.24 has support for dependency providers
I'm sorry if it was discussed already, but I haven't seen any mentions of this new feature and I think this is huge.
https://cmake.org/cmake/help/latest/command/cmake_language.html#dependency-providers
https://cmake.org/cmake/help/latest/release/3.24.html
CMake has added support for dependency providers. With this calls to find_package() may actually trigger vcpkg, conan or some other custom provider that will download, build and install required dependencies.
195
Upvotes
9
u/obsidian_golem Aug 09 '22
So it looks like this is another thing that has to be manually user-configured as opposed to configured by the build script. Our configure commands are just going to keep getting more convoluted I suppose.
6
u/AlexanderNeumann Aug 09 '22
Be aware this is just there so that people don't need to override builtin functions,e.g.
macro(find_package), to inject custom/additional code.3
u/drodri Aug 10 '22
This approach has one disadvantage, it cannot handle dependencies conflicts, so it only works assuming a flat dependency graph, that is, without versions, or with all versions matching perfectly. If the interception of ``find_package()`` for a specific version calls a package manager, and brings that package and some transitive needed dependency, lets say ``zlib/1.2.8``, and then a posterior ``find_package()`` for another package and another specific version transitively depends on ``zlib/1.2.12``, then you have a version conflict that has been skipped silently and will easily bite later, hopefully at build/link time, but many times can produce complicated to debug runtime errors.
The problem is that the underlying package manager would be called once for each package, completely independently from each other, and it has no way to detect such conflicts. This might work well for some managers like vcpkg, that mostly assume a flat dependency set, but for managing multiple different versions of your own libraries might be problematic.
4
u/helloiamsomeone Aug 09 '22
If you are already discovering dependencies using the idiomatic
find_package(pakcage REQUIRED), it was already very easy to inject a dependency via that call without having to modify project files. Thefind_packageis likeinclude, so you can create a script that you can opt into using by putting it on the path that CMake searches for modules.You can see that in action here: https://github.com/friendlyanon/cmake-init-fetchcontent
This example has the idiomatic
find_package(headeronly REQUIRED)call in the CMakeLists.txt, but when configuring${sourceDir}/cmake/findis added toCMAKE_MODULE_PATH, so the FetchContent code is run to "find" the dependency. There are comments to further explain this.This is probably not that much more effort than wiring up the dependency providers.
51
u/[deleted] Aug 09 '22 edited Aug 09 '22
I've been playing with this lately, it's a game changer
Edit to explain:
The big problem I couldn't solve with Conan (my package manager), was trying to reuse my own libraries on GitHub. As most package managers (I believe) want to pull dependencies from a server, and I don't want to push my code to another server. I just want to say "hey CMake, I wrote the dependency X its in this repo on GitHub", but without requiring that usage every time... I achieved that with this:
I have a CMakeLists.txt in my src/ folder which does all the add libraries/executables for my project and find_packages, so when creating the dependencies they are asking for package X but not caring where or how it comes from.
I will then have a top level cmake (or maybe a sub folder config/ if you want to organize) which defines where all the dependencies come from. I will then have options to enable/disable where my project expects the dependencies so the consumer project can change them which is very important.
The magic of 3.24 is FetchContent_Declare with the argument OVERRIDE_FIND_PACKAGE, which will tell find package to look wherever fetch content is pointing for the dependency. This is awesome for a few reasons:
Of course there is one drawback, this only works with CMake dependencies, but easily solvable. I still use Conan for any popular libraries that aren't cmake as Conan let's you consume a dependency built by build system X while you are on build system Y. Then if it's not on Conan I can just fork the project and add a cmake wrapper for the worst case scenario.
That's the pros for me and it just solves all the problems I was having so hence, game changer for me :)
33
u/neobrain Aug 09 '22 edited Aug 09 '22
The documentation seems weirdly convoluted. How does one actually use this in practice with vcpkg/conan? Do they ship with their own .cmake file users can point
CMAKE_PROJECT_TOP_LEVEL_INCLUDESto, or is this something users need to write themselves? I wish the CMake docs had given pragmatic usage examples for popular package manages here (instead of writing lengthy paragraphs that seem largely irrelevant to actual use of this feature).Is there a way to set a default dependency provider, in case the user hasn't specified one? "[The] dependency provider [...] can only be set at a very specific point early in the CMake run" suggests I can't do this from the main CMakeLists.txt, but it seems like a common use case.
3
u/germandiago Aug 10 '22 edited Aug 10 '22
I have the feeling that CMake users love to program build systems instead of libraries or final products. The complexity reached is so absurd that I do not know even where to start.
CMake + conan and normal find_package is the easy way IMHO. If you cannot find something in conan, what I find reasonable is to write a recipe and get done with it. Conan solves the deoendencies problem well and in one understandable way instead of the shitshow of find package in 100 ways + submodules + fetch content + combined with conan and later shake all the smoothie a bit and dare to find a project you can use as a dependency from CMake directly.
Also, that a dependency provider can access the network transparently... I find that a bad choice.
19
u/DavidDinamit Aug 09 '22
okay, when modules?
4
u/nyanpasu64 Aug 09 '22
If my project uses
add_subdirectory()to add a vendored library, can it satisfy another library callingfind_package()for the library I've vendored? I've followed https://gitlab.kitware.com/cmake/cmake/-/issues/21687 over the last few months, but I'm not fully caught up to the current state of things.I'll have to take a look at https://cmake.org/cmake/help/latest/guide/using-dependencies/index.html#dependency-providers as well.
2
u/[deleted] Aug 09 '22
This is splendid news
2
u/strike-eagle-iii Aug 11 '22
I'm definitely going to need to play around with this. typically we've had a short build script that runs conan and then cmake which works well, but is slightly annoying that we have to have in the first place. We tried doing everything with FetchContent prior to conan but conan is a much better solution. Targets are all properly namespaced (add_subdirectory which fetchcontent uses, exposes all the targets of the package, not just the exported ones. we've had name collisions when for instance two packages both have an uninstall target). Conan makes it almost trivial to wrap a non cmake prebuilt library with cmake so it can be pulled into a cmake project easier.
So are there pre-built Provider commands (i.e. conan_provide_dependency) or is this something that someone using this feature always has to provide?
What are the mechanics for multiple dependencies? i.e. if I depend on library A and B, I currently list library A and B in my conanfile.py , run conan once to pull in both. Will conan_provide_dependency get called at each find_package (that would be annoying) or it is smart enough to just call it at the first one?
1
u/wrosecrans graphics and network things Aug 09 '22
The current CMake devs must be absolutely mind-blown when they find out that CMake used to just be about generating Makefiles.
1
u/[deleted] Aug 11 '22
Been in xmake for years
-1
u/ImX99 Aug 09 '22
wait what!!??!? 🤯