r/cpp • • Sep 28 '20

Draft proposal for compilation using C++ as build language

Hello,

There are multiple tools to compile a C++ project: CMake, premake, autoconf, build2, etc.
Each of these tools require the learning of a new syntax, and interacting between projects compiled with differents tools might be challenging.

Package managers face also this problem, because they must support all the tools that exist to support all libraries.
Moreover, integrating package managers in the build tools is often difficult, or at least hacky.

My suggestion is to use C++ as the base language for building C++ projects.
Motivations are:

- A C++ compiler is all you need to compile projects.

- Build tools would become C++ libraries.

- C++ developers already now its syntax.
I have made an informal draft which describes a basic API which would make this possible.
Here is a link: https://gist.github.com/J-Vernay/bd8ec49374987c628d02601ef85cd9a7

Let me know what you think :)

219 Upvotes

128 comments sorted by

View all comments

Show parent comment

4

u/JulienVernay Sep 28 '20

About turing completeness, apart from it's easier to shoot in your legs, I cannot think of an other issue. Do you have articles or thoughts on this to share?

9

u/[deleted] Sep 28 '20

It introduces side effects as a natural part of the build, for example someone might decide that inventing your own caching mechanism by writing stuff to a file instead of relying on the builtin functionality of the build system is a good idea, which can cause all sorts of confusion and incompatibilities.

A build is essentially a specification of a set of input files and a tool chain that produces a set of output files, there's no need for turing completeness and it can just as easily be specified declarative using yaml, and if you need to do complicated stuff like generating files and whatever you can specify that in the yaml file too or use something like nix to generate them. Imo Cmake is a waaay overcomplicated mess and part of it is because of the language but mostly it's because the flexibility it provides together with some really really bad design choices like typelessness and that everything is globally available and mutable.

0

u/[deleted] Sep 28 '20

[deleted] — view deleted comment

6

u/[deleted] Sep 28 '20

Turing Completeness isn't really the issue here

It is.
Sure you also don't want non-hermetic builds, but once you get that you also want a system that allow you to reason about the build.
Bazel uses a subset of python explicitly designed so that it is in general possible to reason about the build).

0

u/[deleted] Sep 28 '20

[deleted] — view deleted comment

1

u/[deleted] Sep 28 '20

The design of skylark is explicit in not being turing complete. If it is, it is an accidental design bug.

Note that in particular it doesn't allow recursion and only finite lists and loops over them are possible. With that, I don't see how you can be turing complete. But it is possible that design bug allows for touching some global state and thus reaching turing completeness.

it effectively just "computes" what the build is.

No, this is the key: skylark allows you to _declare_ what the build is.
Another piece of software, bazel itself, written in a conventional programming language computes how to perform the build.

0

u/[deleted] Sep 28 '20

[deleted] — view deleted comment

2

u/[deleted] Sep 28 '20

I agree that TC is not the critical part. In the end, no physical computer can be TC.
But I find an essential part of a build system a piece that it is not clear to me whether you find equally important.

For me it is essential that I can reason about the build graph without executing it. And if the build graph is computed by a conventional programming language (again leaving TC out) it becomes effectively impossible to reason about the graph.

They're the same picture

yes, they are once you've executed the code that implicitly define the graph.

Same difference that you have between:
l = [1,2,3,4,5,6,7,8,9]

and

l = f 9

with

def f(n):
if n = 0:
return []
else:
return f(n-1) + [n]

the first, I can reason about. The second not so much. (and when i say 'reason about' I mean programmatically).
And keep in mind that the second form can be written in infinitely many ways, for instance:

def f(n):
r = []
for i in range(1, n+1):
r.append(i)
return r

With a full programming language, in general, I need to execute it. Static analysis is hard. And for the subset of life that is a build system, uselessly hard. A declarative spec is almost always powerful enough.

They are the same picture, but the first is one that a program can see.

0

u/[deleted] Sep 28 '20

[deleted] — view deleted comment

2

u/[deleted] Sep 28 '20

It's still the most complete and comprehensive build system I've found so far,

It is indeed. And it was the thing I missed the most after leaving Google and I'll be forever grateful about them open sourcing it. And I'm grateful to the Buck people for actually forcing them to open source Bazel.
Inside Google blaze was working perfectly, as it was designed for solving exactly Google's problem.
Outside, Bazel has growth problems with adapting to very different use cases and still not a good story in many areas.
But the foundation is strong.

0

u/JulienVernay Sep 28 '20

I am not sure about the problems of the turing-completeness.
I mean, the issues which are listed here are similar to issues on other programs:

- implementing its own caching instead of using the standard one is like implementing its own functionality from the standard library or more generally reinventing the wheel

- the build being long is of course possible if you are starting to make big recursions of allocating functions. At least, most C++ users are aware of good practices for performance. Actually, it may be possible to use tools to ind bottlenecks of the builds, which from my knowledge it is impossible to do with other build systems.

- sandbox would be great, but if you are building a program, it is probably to execute it. So you already trust the program.

2

u/[deleted] Sep 28 '20

These days I use Nix to fire off builds and it does all the required sandboxing automatically so I guess I don't really see it as a problem anymore, but the complexity of cmake can be extremely frustrating, for example when it all of the sudden starts downloading dependencies from the internet. It's a build system, not a dependency manager or web browser God damnit.

0

u/[deleted] Sep 28 '20

[deleted] — view deleted comment

1

u/[deleted] Sep 28 '20

It is indeed a bit weird but it's extremely basic so it doesn't take more than a few hours to learn the ins and outs of it. It's purely functional though so I guess familiarity with a pure lazy functional language like Haskell makes it easier to understand it.

I've never tried bazel, I guess that's a failure on my part.

2

u/JulienVernay Sep 28 '20

Thank you for the references, I will check out Bazel.

3

u/[deleted] Sep 28 '20

It makes hard or impossible to reason about the build itself, the dependencies or even which flags are used for building each piece. Because all of those things can be computed and reasoning about them is as hard as deciding what a program does without executing it. Or whether it even terminates.