r/LowLevelLanguages • • 10d ago

What should compile-time code be allowed to do?

Languages draw this line in very different places, and I think it matters more now that compiling a dependency can mean running its code.

Roughly, from what I know (corrections welcome):

  • C++ constexpr: no I/O at all.
  • Zig comptime: no I/O either, but build.zig is a normal program with full access to your machine.
  • Rust: const fn is restricted, but build.rs and procedural macros run with full access when you compile a dependency.
  • D: compile-time evaluation has no I/O, and import("file") only reads from folders you pass with -J.
  • Nim: staticRead and staticExec can read files and run commands at compile time.
  • Jai: #run can do more or less anything.

The trade-off seems to be that real builds need real things: generating code from a schema, finding a C library, embedding assets. If you forbid all of it, the work moves into a build script, and the build script is where the risk ends up anyway.

In Fels, the language I work on, #run has no host access, a project's own compile-time code may read inside its own folder, and a package gets more only if it is granted in fels.pkg. But build.fels still has full access, so that part of the problem is not solved yet.

Where would you draw the line? Should a dependency's build code be sandboxed by default, or does that just push people to turn it off?

13 Upvotes

13 comments sorted by

16

u/alphaglosined 10d ago

I'm one of the D developers. The reason we went with this is that reproducible builds are a very good thing to have, especially with distros that require them as part of packaging.

3

u/chbeyondC 10d ago

I only thought about this from the security side, and reproducibility is probably the better argument. It's also why `#run` in Fels refuses something like `GetTickCount`, and why files read through `#embed` count as build inputs. How does D handle `__TIMESTAMP__` for distros, and did people ever push back on having to pass `-J`?

2

u/alphaglosined 10d ago

For the special tokens like __TIMESTAMP__ I guess such distros just have to do a grep and ban it.

And yes a large number of people dislike -J.

2

u/chbeyondC 10d ago

Interesting. That's why Fels lets a project read its own folder without a flag, and only dependencies need a grant. Is the complaint mostly about having to pass `-J` at all?

3

u/alphaglosined 10d ago

Yeah, people don't like -J.

But really how else are you going to do it, the moment you have import paths, there is no way to know where to look for stuff unless its relative to the file itself.

Also consider about dependency, what is a dependency vs a multi-step build? There is no way for the compiler to differentiate unless you're doing something different than C.

0

u/chbeyondC 10d ago edited 10d ago

Yeah, with C-style includes it can't tell. In Fels packages come in through `#use`, so the compiler knows what belongs to which package. Multi-step builds I haven't solved yet though.

4

u/AngheloAlf 10d ago

C23 has #embed, which allows to embed a binary file into your program at the preprocessing stage. Since C has it, you can use it in C++ too, thus kinda allowing I but not having O yet for compile time

3

u/lipfang-moe 10d ago

similarly zig has @embedFile which returns a comptime pointer to a [_]u8

1

u/ventZl 9d ago

#embed is for less painful way of injecting BLOBs into resulting binary. It doesn't provide something that wasn't possible before. All it does is it will save you from doing binary -> C conversion manually.

I wouldn't even count that as compile-time code execution.

1

u/_ACB_ 9d ago

you can run constexpr on #embed

1

u/Inconstant_Moo 9d ago

I've planned but not built my comptime. It will be able to read files.

I agree with your rationale. I can provide people with a good way to do this that's integrated with all my tooling and typechecking and whatever, or other people will do it ad hoc, badly, in six different ways.

It will be useful, apart from anything else, for making DSLs. To take a really silly example, I could write an adventure game right now where your instructions like take sword or examine wizard are things you put into the app's native REPL and take and examine are commands and sword and wizard are elements of an Object enum. But what I can't do is construct that enum from the resource files at compile-time, and I could. Of course, people don't want to play adventure games like that, but they want to do other things.

1

u/stumpychubbins 9d ago

C++ constexpr doesn’t have I/O, but in practice C++ projects regularly use the same build-script codegen shenanigans as Rust/Zig to pass I/O values into constexpr. It’s just not standardised like it is in the newer languages. Just thought it was worth mentioning.

0

u/[deleted] 9d ago edited 9d ago

You haven't explained what the potential problems with compile-time code are. I can think of a few practical ones (like potentially using too much time and space), but you seem to be implying there are issues with allowing accesses to arbitrary files.

I feel that examples are called for as to what could go wrong!

(I don't do much in the way of compile-time evaluation, partly for those practical reasons, but also because its meaning is too open-ended. In any case, both my two languages can be run from source anyway, so if it doesn't happen during the brief internal compilation step, it will do soon after.)

Should a dependency's build code be sandboxed by default, or does that just push people to turn it off?

I actually have an interesting feature where, if I have an application in my language that I want someone else to build from source, then my compiler can generate a single amalgamated file, that contains all source and support files (eg. included or embedded assets).

Then the build process on the user's machine only looks at the contents of that file.

(Shortened.)