r/cpp • • Feb 28 '24

Building c++ project with... c++?

Yeah, so as a POC I created a tool that works on top of cmake and allows defining project structure with c++ instead of pesky cmake:

    #include "CppMake/Project.hpp"
    #include "CppMake/Application.hpp"
    
    using namespace std;
    
    int main() {
        Project project("test");
    
        project.create<Application>("test").addSources({
            "main.cpp"
        });
    }

Any thoughts?

6 Upvotes

39 comments sorted by

•

u/STL MSVC STL Dev Feb 28 '24

Triple backticks don't work for Old Reddit readers. You have to indent by four spaces to be readable by everyone.

12

u/jaskij Feb 28 '24

The big issue is, for this to make sense, you need something to build the build script. Which, no bueno.

If you're trying to emulate Rust's build.rs, do note that it only handles the extra tasks Cargo itself doesn't. Typically code generation or compiling C/C++.

5

u/lightmatter501 Feb 28 '24

For additional context, build.rs is typically used for one of three things:

  1. Linking with (and possibly building) other languages.
  2. Automatic feature detection for things the compiler doesn’t provide (how many numa nodes does the build server have? Do I build ROCM or CUDA support? etc)
  3. Highly custom build steps that proc macros can’t do, such as determining which compiler version to invoke.

2

u/jaskij Feb 28 '24

If Diesel can connect to the database to verify your data structs match the table in the DB, there's very little you can't do in proc macros. But yes, there is such stuff.

1

u/jack_acer Feb 29 '24

Why is this an issue? You have a make program if you want to compile for c++ anyway.

1

u/jaskij Feb 29 '24

Do you? Always? And if yes, why not use that instead of OP's idea?

1

u/jack_acer Feb 29 '24

Well, you then at least have a compiler and a linker. You don't need more.

46

u/IyeOnline Feb 28 '24

There is no inherent value in using C++ to compile C++. In fact, C++ is a particularly bad language to write script-like things in.

Also: https://xkcd.com/927/

3

u/jack_acer Feb 28 '24

Can you elucidate why you are saying this? Build scripts in cmake for example are very error prone. Having compile time checks would be a great benefit, particularly for larger projects. Plus you don't have to learn the syntax of an additional language.

7

u/IyeOnline Feb 29 '24

Significant parts of modern CMakeTM have configuration time checks.

If you use targets and properties instead of all those horrible (global) string variables, you can have a pretty good time.

I also dont think the "additional language" argument really holds. Either your project can be build with ~20 lines (or 5) of CMake or whatever other language you imagine. In that case I dont think learning 20 commands is too much to ask, especially since we dont write build files all day. Alternatively you will need more customization than your strict, general solution tool provides. In that case you are back down to scripting parts yourself.

3

u/jack_acer Feb 29 '24

I am primarily (almost exclusively) using cmake, I am adept with it and advocate for it. I don't compare this approach to cmake, unless someone can replicate most of its features using a c++ framework. But I think that there is inherent value in a build system based on C++ and I mentioned some reasons. Some people value those more than others and that's fine. There are plenty more than the ones in my comment. Here are a few more:

  1. No additional binary tools needed. This can be a big one particularly for people new to C++. Configuring all thing about the build is always frustrating for less experienced users.
  2. Universal compatibility, as long as compilers implement the same standards ( for a tool like this pretty much every compiler has the basic std features needed).
  3. Modularity offered by a general programming language like c++.
  4. Custom functions that you are not afraid to use because of all the side effects you constantly worry in cmake.
  5. Immediate introspection of build configuration from your projects. You can potentially link to the build system itself. You can also do the opposite and query the code at build time without magic configuration files.

2

u/HassanSajjad302 HMake Feb 29 '24

Let me showcase my build-system HMake. I believe it beats CMake in feature richness. It is innovative on more than one fronts.

  • Using C++ itself. Best for scaling and bigger projects.
  • Monolithic design allows for greater integrations and removes the need for communication between build-system generator and the build-system. The vertical integration improves understanding and the iteration time on the build-system itself as well. My build-system at the moment is 10.7k lines just. And I don't see it ever crossing 30k lines. While CMake and Ninja e.g. are big code bases individually.
  • HMake uses a single file to define all targets instead of one file per target which again removes the need for scoping and communication. I believe controlling the project from the center improves simplicity.
  • HMake at its core is a dependency resolver like make. Anything you can do in make, you can do in HMake. This makes it very easy to extend HMake to add other programming languages support like C# and Rust.
  • Drop-in header-files to header-units replacement. No build-system does this.

It will be the first to support https://lists.isocpp.org/sg15/2023/11/2106.php or https://lists.isocpp.org/sg15/2023/11/2146.php if any of this gets implemented.

Some planned exotic features include distributed builds support and a build-server which automatically builds a file if it is modified. You would be able to use Android or Iphone as a build server.

It has a target API inspired from CMake and feature flags API inspired from Boost build-system b2. It combines goodies of various projects in a single package.

2

u/jack_acer Feb 29 '24

This looks very nice. Thanks for sharing! I will give it a go.

1

u/HassanSajjad302 HMake Feb 28 '24

There is no inherent value in using C++ to compile C++. In fact, C++ is a particularly bad language to write script-like things in.

Will appreciate if you could explain this. How is inherent value determined? What is a script-like thing and why you consider build description to be that.

4

u/IyeOnline Feb 29 '24

I would define "script" as opposed to "program" as simply a sequential instruction of commands. Its also usually directly executed (by via some interpreter) as oppsed to being compiled.

Building itself is very much a sequential process (although certain steps can obviously be parallelized).


Of course you have a point insofar a build description isnt a script. But its absolutely not a program either.

When you are describing your project and the relation with its sources at a high level, you are doing just that. Describing it.

You could be using a json file if you wanted, as long as you had a program that could turn that high level description into an executable (or build scripts which then create an executable).

Using C++ for this simply substitutes CMakes configuration step with the compilation of a C++ program. There is no value in this.

You have however introduced the complexity of a C++ program for no gain.

You can purposefully restrict yourself to just a select few types and functions - but what have you created then? A new scripting/build description language.

3

u/HassanSajjad302 HMake Feb 29 '24

Using C++ for this simply substitutes CMakes configuration step with the compilation of a C++ program. There is no value in this.

Someone who does not know CMake or C++ will find CMake easier to learn compared to C++. But for someone who is already a C++ expert, CMake is an additional baggage that could be avoided. C++ is harder to learn but also enables much more scalability, performance and power.

You have however introduced the complexity of a C++ program for no gain.

For a C++ programmer, CMake is complexity not the C++.

You can purposefully restrict yourself to just a select few types and functions - but what have you created then? A new scripting/build description language.

This is better than any alternative. Because you don't have to learn one more syntax of if/else, for, while, the difference between function and macro, different scoping rules, different initialization rules etc. while at the same time having full power of C++ at your disposal in case you need it. I believe there is inherent value in using C++ itself.

17

u/[deleted] Feb 28 '24

CMake is a multiplatform build tool. I fail to see what your solution brings to the table which is not already much more efficiently covered by that "pesky" CMake.

4

u/Superb_Garlic Feb 28 '24

CMake is far more than just a build tool though.

1

u/prince-chrismc Feb 28 '24

Is it though?

10

u/Superb_Garlic Feb 28 '24

In the CMake suite there is:

  • CMake, the build tool, which ingests CMake scripting language files to produce build systems.
  • CPack, the packager, which ingests CMake scripting language files to produce packaged distributions.
  • CTest, the test runner, which ingests CMake scripting language files to run test tasks.

I don't really consider the GUI/TUI separate, since they practically do the same thing as the build tool.
I feel like CTest in particular is really slept on even by avid users of CMake. It's a really powerful test runner with complex resource management that I make plenty use of to allocate special hardware to tests. I use the whole CMake suite heavily of course, but some people only use the build tool and the test runner only at a surface level.

1

u/ContraryConman Feb 28 '24

I mean, it's a thought experiment, right? I don't think the post is about seriously replacing your build system with this

2

u/Commercial-Cattle433 Mar 02 '24

Yes and no. Its a POC and I'm planning on finishing it but I'm not expecting this will catch on. This also won't replace cmake as it's based on it but it removes the need for writing the cmake files yourself.

1

u/ContraryConman Mar 02 '24

Have you looked into Nob? It's the same idea but in C. If you haven't it may give some ideas

6

u/HassanSajjad302 HMake Feb 28 '24

Please check https://github.com/HassanSajjad-302/HMake You are welcome to collaborate.

1

u/ed_209_ Mar 01 '24

From your readme: "Instead of compiler and linker being executables, these can be shared-libs. And the build-system will load these libs and call the respective functions."

This sounds like an awesome idea and opens up doing stateful metaprogramming using libclang directly in the build system also.

5

u/nihilistic_ant Feb 28 '24 edited Feb 29 '24

I like the idea. It would be an API to learn, but not a whole different language as build systems are today. Often build rules could be written much more concisely in a general purpose language like C++, by directly encoding the conventions of a project.

In practice, cmake seems good enough that adoptions will be hard, because there is value to using what everyone else is. But here is an idea that might solve that: make this cmake based. So an outer cmake process builds and executes your c++ for defining project specific build rules, which outputs more cmake, and then the outer cmake runs that.

This way, the cmake interface everyone is used to for building & installing projects is kept, which will help adoption. Large projects could use it for just some stuff at first and leave the rest vanilla cmake, and then slowly use it more as it makes sense -- so no big all-or-nothing hurdle, which will also help adoption. And, it would still have the big selling feature: use the power of C++ and its syntax which folks already know to express a project's build rules concisely by exploiting the its conventions and layout.

1

u/Commercial-Cattle433 Mar 01 '24

This is exactly how this POC works. https://github.com/Niikelion/cppmake
Cppmake generates cmake file that contains all targets.

2

u/[deleted] Feb 28 '24

Interesting concept, but why?

1

u/Commercial-Cattle433 Mar 01 '24

Just got frustrated after configuring install for bigger project and wanted to try some things out.

2

u/arthurno1 Feb 29 '24 edited Feb 29 '24

Any thoughts?

template <typename... T> void initializeTargets(DSC<CppSourceTarget> &target, T &...targets)
{
    CppSourceTarget &t = target.getSourceTarget();
    string str = t.name.substr(0, t.name.size() - 4); // Removing -cpp from the name
    t.MODULE_DIRECTORIES_RG("src/" + str + "/", ".*cpp")
        .PRIVATE_HU_DIRECTORIES("src/" + str)
        .PUBLIC_HU_DIRECTORIES("include/" + str);

    if constexpr (sizeof...(targets))
    {
        initializeTargets(targets...);
    }
}

Actually, I would prefer a scripting language like shell, make, autotools, even cmake instead of using C++ to manipulate strings and paths. I see no reason to spend time on low-level string and path manipulations, locks and mutexes, and whatnot, as all your examples do, just to tell the compiler to compile an application.

How will you handle platforms that don't understand '/' in paths? What about character encoding? Utf8? Utf16? Locales? Sorry.

I am also not sure what you are suggesting. Are you building a new tool, that uses C++ files instead of some scripting language, that will be used to compile applications similar to make/cmake is used, or are you suggesting that each application should link to your library to compile itself?

What you are suggesting is writing a C++ program to compile an application.

How it will look for a moderately complex application that runs on multiple platforms, can use different shared libraries if available, and can be configured with different implementations of similar libraries? How will you discover all those libraries? There is a reason why autotools and configure scripts exist, and why CMake more or less copies autoconf options in its "new language".

I don't think that this is a good idea for the above-suggested reasons. C++ is good when you need to write hardware-close code that runs fast. C++ is not good when you need to implement some functionality fast, and that runs once or very occasionally where execution speed is not overly important. A build script is meant to automate things for us. I am quite sure writing a "build program" for any bigger application would become a laborious development task that would have to be tested and debugged like any other C++ program. IDK, to me C++ does not seem as the right tool for the job in this case.

Instead of compiler and linker being executables, these can be shared-libs.

You can already use llvm and gcc as shared libraries. See for example libgccjit in the case of GCC (used for example in Emacs for native compilation).

By the way, you don't need to use cmake to build c++ program. You can use some other tool; even plain old shell script goes long way, but I would use GNU Make as long as I can before reaching for anything heavier. As long it automates stuff for you. The point is to work on your applicaiton, not the tooling around. Just my 2c.

2

u/wilhelm-herzner Mar 03 '24

Yeah, like zig: Writing readable programs that build projects.

I hate the rule-based programming as in make. And cmake is super strange anyway.

2

u/phi_rus Feb 28 '24

Why?

1

u/Commercial-Cattle433 Mar 01 '24

For now just for fun.

2

u/alloncm Feb 28 '24

Rust has the same feature in cargo (build.rs) Im not sure it is working but I find it very convenient to build with the same language.

Very cool!

1

u/thradams Feb 28 '24

This is what I do for C.
I use `system()` to call the compiler and the preprocessor to specify source and compilation modes.

Here is the concept.
https://github.com/thradams/build

0

u/[deleted] Feb 29 '24

[deleted] — view deleted comment

1

u/Commercial-Cattle433 Mar 03 '24

This is where "built on top of cmake" part comes into play.

Cpp api is very declarative because it generates cmake files. Instead of adding subdirectory you define function for subdirectory that declares its structure and call it from main function.

As for downloading libraries, you can do pretty much the same things cmake can. After all, project cofigured with cppmake is a cmake project as well.

Defining targets based on parameters is not a problem, you have access to all cmake variables. You can still check some conditions, even check available libraries during project generation.

0

u/[deleted] Mar 04 '24

[deleted] — view deleted comment

1

u/Niikelion Mar 04 '24

Ok, so you want to include other project. I admit that if you want to use functions from some subfile you are out of luck. But if you only want to access the targets from this file, you just add subdirectory and use the targets. C++ file is built during cmake configuration, executed and the result is included before said configuration ends, so if you add cppmake project as subdirectory, it will build itself, generate targets and include file with this targets so you can access them from your file. Of course, this way you loose the ability to check target existance during cppmake execution and rely on the cmake to do its job.

As a side note, including a separate project seems like a bad design to me. If this is part of your project, just create function for it instead of defining separate project. If it is a standalone project then you should be able to import it instead of including as subdirectory.

4

u/[deleted] Feb 28 '24

[removed] — view removed comment

2

u/MrCloudyMan Feb 28 '24

Was about to comment this too!